Three different levers for squeezing more throughput out of LLM serving on AMD Instinct MI355X - speculative decoding, parallelism strategy, and KV-cache quantization. Each tackles the problem from a different angle, and each comes with its own real benchmark numbers.
Quick reference table
| Blog | Key result | Who it’s for |
|---|---|---|
| DFlash Speculative Decoding on AMD Instinct MI355X: Up to 5× Faster Qwen3.5 Inference | Up to 5.02x single-request throughput on Qwen3.5-27B vs. standard decoding, plus 10-20% more when the target model is MXFP4-quantized | Anyone serving large models who’s hit a single-request latency wall |
| Serving GLM-5.2-MXFP4 on AMD Instinct™ MI355X: When Prefill Context Parallelism Pays | Maps exactly where prefill context parallelism helps vs. hurts by prompt length on 4x MI355X - same checkpoint, same hardware, CP on vs. off | Teams tuning multi-GPU serving configs for long-context prompts |
| UltraQuant on AMD Instinct: More Efficient Agentic Serving for Qwen3.8-MXFP4 | 29% more tokens/sec and 24% lower per-token latency vs. 8-bit KV at concurrency 32, with 1.7x faster first token; accuracy within noise on GPQA-Diamond/SWE-bench Lite | Teams needing more concurrency headroom without an accuracy hit |
Start with DFlash if your bottleneck is single-request decode latency, bring in the GLM-5.2 prefill CP findings once you’re scaling serving across GPUs, then layer UltraQuant’s KV-cache quantization on top once you need more concurrency without burning accuracy.
DFlash Speculative Decoding on AMD Instinct MI355X: Up to 5× Faster Qwen3.5 Inference
DFlash replaces the usual autoregressive speculative drafter with a block-diffusion model that denoises an entire draft block in a single forward pass instead of token-by-token. On Qwen3.5-27B this delivers up to 5.02x single-request throughput over standard decoding, with consistent wins over Multi-Token Prediction (MTP), and an extra 10-20% when the target model itself is MXFP4-quantized.
Serving GLM-5.2-MXFP4 on AMD Instinct™ MI355X: When Prefill Context Parallelism Pays
Splitting prompt tokens across GPU ranks during prefill (context parallelism) isn’t a universal win - this piece pins down exactly where it helps versus hurts when serving GLM-5.2-MXFP4 on 4x MI355X, same checkpoint and hardware, CP on vs. off. It maps where the tradeoff sign flips by prompt length, and where single-stream serving already breaks even.
UltraQuant on AMD Instinct: More Efficient Agentic Serving for Qwen3.8-MXFP4
Pairs MXFP4-quantized Qwen3.8-2.4T-A95B with UltraQuant’s 4-bit KV cache quantization on vLLM. At concurrency 32 it serves 29% more tokens/sec at 24% lower per-token latency than the fastest 8-bit KV backend, with 1.7x faster time-to-first-token, and accuracy on GPQA-Diamond and SWE-bench Lite holds within noise of the 8-bit baseline. Multi-Token Prediction stacks on top for further gains at moderate concurrency.
Running any of these on your own workloads? Drop your numbers below!
