single-gpu-llm-toolkit
A full-parameter LLM fine-tuning toolkit built and battle-tested on AMD MI300X — because most fine-tuning tools assume either a multi-GPU cluster or that you’ll settle for LoRA. I don’t think either should be true.
The problem this solves
Most LLM fine-tuning tooling in the wild is built CUDA-first and multi-GPU-first, then ported to AMD as an afterthought — if at all. And most of it reaches for LoRA the moment memory gets tight, which means you’re not actually training the model, you’re training a cheap approximation of it.
I built something different: a toolkit that does real, full-parameter fine-tuning — no LoRA, no adapters, anywhere — that scales cleanly from a single GPU up to a full multi-GPU cluster with the same codebase, same flags, no rewrite. Windowed layer-freezing plus gradient checkpointing gets full-parameter training to fit on a single constrained card; the exact same train_cpt.py scales up to a cluster via --ddp/--fsdp with nothing more than a torchrun launch command. One tool, any hardware you actually have.
Why this matters for AMD specifically
I built and ran this entirely on an MI300X, and it wasn’t a drop-in port from CUDA — ROCm has real, sharp edges that most tooling never surfaces because it was never tested against them. In the process of building this, I found and fixed a genuine GPU architecture misdetection bug: the standard method for reading AMD hardware IDs off the KFD interface was silently decoding MI300X’s real hardware ID into the wrong architecture string — not an error, just quietly wrong behavior on hardware people are trusting to be handled correctly. That’s now fixed, tested, and shipped.
I also hit and solved a real infrastructure problem specific to running training on rented cloud GPUs: bulk data transfer over the standard remote-access path was throttled to a crawl (a 12GB model checkpoint would’ve taken 100+ hours), which I fixed by routing through the cloud provider’s own storage backend instead — cut that down to under an hour. And when a training box’s underlying instance died mid-run, I turned that into a fixed, verified recovery pattern so it can’t happen the same way twice.
None of this is theoretical. It’s what actually happens when you run real training on real AMD cloud hardware for real, and most tooling has never been pushed hard enough to find any of it.
What I’m bringing to the table
- A working, tested, full-parameter fine-tuning toolkit that runs natively and correctly on AMD hardware — not ported, built for it.
- A track record of finding and fixing real ROCm-specific correctness bugs, verified against actual AMD hardware behavior, not assumed.
- An engineering discipline that treats “it looks right” as insufficient — every fix here is backed by a real reproduction and a test that would catch it again.
- Active work extending that same discipline upstream — I’m currently contributing a verified ROCm compatibility fix to a major open-source inference project.
What I’m asking for
Continued access to AMD developer cloud compute to keep pushing this further — scale up the models this handles, harden the multi-GPU path with real cluster runs, and keep finding and fixing the class of bug that only shows up when someone actually stresses AMD hardware instead of assuming it behaves like NVIDIA’s. I’d rather be the person finding these bugs for you than the person who ships around them.