Scaling Out on AMD Instinct: Train the MoE, Wire Up the Cluster, Close the RL Loop

Training a big MoE, or running RL on one, stops being a single-GPU problem very quickly. These three recent posts cover the multi-node stack on AMD Instinct from the bottom up, from the training engine to the orchestrator to a full RL loop running across nodes.


Quick reference table

Blog Key result Who it’s for
MoE Training Optimization with Primus Fused MegaMoE is worth 1.40x (BF16) and 1.81x (MXFP8) on the MoE layer; DeepSeek-V3 reaches 1.095x baseline throughput on 1024 MI355X GPUs Teams pre-training or fine-tuning MoE models at cluster scale
ROCm 10.0.0 Support for Ray 2.58.0: Enabling Scalable AI Workflows Existing Ray code runs on Instinct unchanged, validated with PyTorch 2.12 and vLLM, including a tensor-parallel Ray Serve LLM endpoint Anyone already orchestrating training or serving with Ray
DeepSeek-V4 Flash RL on AMD Instinct MI308X GPUs with an AITER-Backed RL Stack End-to-end GRPO run on 24 MI308X GPUs: AIME-2024 accuracy climbs from 30.0% to 49.2% in 75 steps Practitioners building RL post-training pipelines for large MoE models

Read in order, they build up the stack: make the MoE training step fast and keep it efficient as you add nodes, let Ray handle placement and scaling without vendor-specific glue, then see a full RL loop where a training actor and an FP8 rollout engine share one cluster.


MoE Training Optimization with Primus

A follow-up to AMD’s MoE training best-practices post. It walks through the optimizations built into Primus for MI355X: FP8/MXFP8 expert GEMMs, and a fused MegaMoE kernel that hides the all-to-all inside the grouped GEMM. The scaling section is the part to bookmark. With a tuned pipeline layout and per-scale recompute settings, DeepSeek-V3 runs at 1.095x baseline on 1024 GPUs, and 128 nodes keep 97.7% of the 32-node per-GPU throughput. It also covers Primus Projection for planning a run before you launch it, and the JAX (MaxText) MoE path.


ROCm 10.0.0 Support for Ray 2.58.0: Enabling Scalable AI Workflows

Ray’s selling point is that placement, scaling and fault tolerance live in framework APIs rather than device code. This release keeps that true on Instinct: Ray finds the GPUs automatically, torch.cuda maps to ROCm, and your existing Ray code runs as-is on a stack newly validated with PyTorch 2.12, vLLM and current SKUs up to MI355X. The second half is a hands-on example that stands up an OpenAI-compatible, tensor-parallel LLM endpoint with Ray Serve, in which exactly one line names AMD hardware.


DeepSeek-V4 Flash RL on AMD Instinct MI308X GPUs with an AITER-Backed RL Stack

This is the whole loop, not an isolated training step. Three MI308X nodes run DeepSeek-V4 Flash RL as disaggregated services: a BF16 MoE actor on two nodes, and an FP8 rollout engine on the third that frees up memory for KV cache and long generations. Ray coordinates both groups, and weights sync over RDMA online. Over the first 75 GRPO steps, AIME-2024 accuracy rose from 30.0% to 49.2%. The post also covers evaluation and checkpoint recovery, which are the parts that usually get skipped.

Drop questions or your own scaling numbers below - always good to see how these setups hold up on different cluster sizes.

Amazing collection! I appreciate the detailed walkthrough