Sometimes the honest result of an AI hardware experiment is “the GPU didn’t help.” This is that post.
Most of what we’ve published about this hardware has been about Vulkan acceleration winning. This one is different: we tried to accelerate voice cloning on the RX 580, and the GPU path lost to plain CPU. We’re documenting it anyway, because the failure mode here — and the five bugs that nearly buried the project before we found that out — are exactly the kind of thing that wastes a weekend if nobody writes it down.
The pipeline
Goal: a production voice pipeline for a YouTube channel, separating performance from vocal identity:
Script → Balabolka + Microsoft Neural TTS (acting/prosody) → WAV
→ Applio/RVC (vocal identity transfer) → Final cloned voice
The TTS engine handles emotional delivery and correct pronunciation of technical terms. RVC overlays a trained voice identity on top. The result is consistently more natural than asking a generative TTS model to do both jobs at once — 80–95% naturalness vs. 60–70% for TTS alone, by our own listening tests.
Hardware: Intel Xeon E5-2690 v3 (24 threads) · AMD Radeon RX 580 2048SP (8GB VRAM) · 31.8GB DDR4 REG ECC · Windows 11 · Python 3.11.9
Failure 1 — the dataset that ate its own memory
Original plan: train RVC on hours of source TTS audio, auto-sliced into 6–12 second clips via Whisper. Result:
MemoryError: concurrent.futures.process.BrokenProcessPool
~2.5 hours of source audio, sliced into 1,000+ clips, blew RAM and VRAM simultaneously during preprocessing. Even on the runs that completed, that many clips destabilized the rest of the Applio pipeline.
Fix: abandon the large dataset. Use 9 minutes of clean TTS audio instead. Smaller, cleaner, far more controllable — and apparently enough; see training results below.
Failure 2 — Python version + a CUDA line that silently poisons Windows
Applio requires Python 3.11+ (numpy 2.4.4 has no 3.10 build). Installed alongside the existing 3.10 via py -3.11 -m venv venv — no need to remove anything.
The real trap was in requirements.txt:
torch==2.7.1+cu128; sys_platform == 'linux' or sys_platform == 'win32'
+cu128 is CUDA 12.8, NVIDIA-only. The condition sys_platform == 'win32' is always true on Windows — so this line installs a CUDA-only build on a machine with no NVIDIA GPU, every single time, with no error until much later. Fix was to strip the +cu128 suffix from torch, torchaudio, and torchvision entirely.
Failure 3 — DirectML worked. It just didn’t matter.
With Applio running stable on CPU (~10 min/epoch), we tried accelerating with DirectML:
import torch_directml
dml = torch_directml.device()
print(dml) # privateuseone:0 — RX 580 correctly recognized
Real measured result:
| Metric | CPU | DirectML |
|---|---|---|
| GPU utilization | — | 7–10% |
| VRAM used | — | 1.2 GB |
| Speed | 4.46 s/it | 4.27 s/it |
~5% difference. Statistically irrelevant. Worse: torch-directml is frozen at 0.2.5.dev240914 (Sept 2024) and hard-requires torch==2.4.1, while Applio needs torch==2.7.1. That’s an unsolvable dependency conflict, not a configuration problem:
torch-directml 0.2.5 requires torch==2.4.1
hyper-connections 0.4.11 requires torch>=2.5 ← conflict
Microsoft has stopped meaningfully investing in DirectML for AI workloads. Conclusion: DirectML for RVC training on AMD Windows in 2026 isn’t worth the dependency cost for the gain it delivers. We dropped it entirely.
Failure 4 — the silent fake training run
While forcing CPU fallback, we tried setting these environment variables:
# ❌ NEVER do this with Applio on AMD:
set USE_DIRECTML=0
set CUDA_VISIBLE_DEVICES=-1
set ROCM_VISIBLE_DEVICES=-1
set HIP_VISIBLE_DEVICES=-1
These silently broke feature extraction. Training would finish in seconds, report “Model trained successfully,” and produce no .pth file at all:
Not enough data present in the training set. Feature files in logs\my-project\extracted could not be loaded correctly.
The trap to watch for: if training reports success in seconds instead of hours, it didn’t train anything. Always verify .npy files exist in logs/<project>/extracted/ before trusting a “successful” run.
Failure 5 — missing silence files
A fresh git install doesn’t ship the silence/mute reference files RVC expects during training. They surface one at a time, mid-training:
FileNotFoundError: logs\mute\extracted\mute.npy ← wrong shape crashes training
RuntimeError: tensor (100) must match size (768)
Fix: generate all of them up front, with the exact shape the contentvec embedder expects — (196, 768). Any other shape crashes the run.
What finally worked
Clean venv, torch 2.4.1 base (no DirectML), dependencies installed with --no-deps to avoid torch getting silently upgraded, config.py defaulting to "cpu" when CUDA is unavailable, mute files pre-generated, GPU number field left blank in Applio (not 0, not -1, not cpu — blank).
Training results — 9 min source, 200 epochs, batch size 4:
| Epoch | Loss | Time/epoch |
|---|---|---|
| 1 | 27.181 | 7:21 |
| 8 | 17.93 | 7:02 |
| 20 | 16.212 | 7:43 |
| 43 | 15.785 | 5:43 |
~6 min/epoch average. ~20 hours total for 200 epochs, fully on the Xeon’s 24 threads. Inference after training: ~5–30 seconds per minute of audio — a 2-hour batch processes in roughly 30 minutes.
Verdict
| Platform | Status |
|---|---|
| NVIDIA CUDA | Excellent — native support |
| AMD ROCm (Linux) | Workable with effort |
| AMD Windows, DirectML | Abandoned — real but irrelevant gain, unsolvable dependency conflicts |
| AMD Windows, CPU-only | Functional, stable, slow, and the only option that didn’t fight us |
The honest finding here isn’t “Vulkan wins again.” It’s that for this specific workload, on this specific hardware, the right call was to stop chasing GPU acceleration and let the 24-thread Xeon do the work. Sometimes the correct optimization is recognizing where the ROI isn’t there.
Thanks to this community — and specifically to POIZONE and Vector.sys — for the access and the space to share this.
Full pipeline, scripts, and the step-by-step Cookbook for running local AI on legacy AMD/Nvidia/Intel hardware via Vulkan (supported since 2016/2017): GitHub - aivisionslab-studios/rx580-local-ai-guide: Complete guide to running local AI on AMD RX 580 8GB via Vulkan — llama.cpp, Ollama, OpenWebUI, Stable Diffusion. No CUDA. No cloud. Free. · GitHub
Hardware doesn’t die — it just waits for the right software.
#ai #amd-tools #voice-cloning #rvc #applio #tts #windowscanonical