Hi, I´m developing a fork of llamacpp and it is the fastes decode for Qwen 3.8 next.
It will be avalible on Git next week on ftoleedo · GitHub
1 Like
Welcome to the community! What optimizations are driving the decode-speed gains over upstream llama.cpp? Looking forward to seeing the benchmarks and code when it’s released.
1 Like
I have bean working on seral sides:
MoE experts (prefill)
hipBLASLt solution counts
Timing on gfx115
Hardware counters
Hyper-connections
Dense projections (prefill)
and orthers.
I will be post a binary at first next week, I’m still fine-tuning the prefill