Technical Dive into AMD MLPerf Inference v6.1 Submission - the kernel-level tuning behind AMD's biggest MLPerf scale-out yet

What’s in it: AMD’s fifth consecutive MLPerf Inference round pairs the new MI350P PCIe GPU with MI355X across GPT-OSS-120B, Llama2-70B, DeepSeek R1, Wan2.2-T2V-A14B, and DLRM v3, and walks through the actual optimization work behind the numbers - MoE kernel tuning, FP8-to-MXFP4 quantization, attention kernel optimizations, and vLLM/SGLang scheduler tuning, each with its own measured gain.

Why it matters to you: If you’re trying to squeeze more throughput out of large MoE or multimodal models on Instinct GPUs, this is a rare look at which specific techniques - and how much each one contributed - actually moved the needle in a validated, third-party-audited benchmark.

Key result: A 512-GPU configuration - the largest-scale submission in MLPerf Inference history - delivered the highest total LLM throughput ever recorded in an MLPerf round.

Read it here: Technical Dive into AMD MLPerf Inference v6.1 Submission — ROCm Blogs