Can one MI300X run an entire small factory? Our "AI Factory" math for MSMEs — and where the community would attack it

I’m the founder of AiVibe — bootstrapped, building an “AI Factory” platform called AiAmbA for small and mid-sized manufacturers, and currently onboarding onto the AMD Developer Cloud (MI300X). This is a genuine engineering discussion, not a product pitch — I want this community to attack our assumptions before physics does.

THE PREMISE WE’RE BUILDING ON
Most of the world’s factories have 2–50 machines and will never afford hyperscaler-style automation. Our bet: enrich 100+ categories of existing industrial devices through their I/O pins into “AI-brained” machines, run perception and reasoning on cloud GPU (MI300X) + on-device edge inference, and let the owner operate the plant in natural language. The economic target that makes or breaks it: one AI agent covering three shifts at roughly the cost of a single operator’s eight-hour wage.

WHERE I’D GENUINELY VALUE THIS COMMUNITY’S JUDGMENT

1. Single-GPU 70B serving as the cost anchor.
The 192 GB HBM3 on MI300X lets us serve quantized 70B-class open models (FP8/AWQ, vLLM on ROCm) on ONE GPU — no tensor parallelism, no interconnect tax. For those running vLLM on MI300X in production: what real tokens/sec and batch-depth are you sustaining, and where does quantization start costing you factory-relevant accuracy (structured tool-calls, not prose)?

2. Day/night utilization.
Our plan runs interactive voice-agent serving by day (our AiVedhA voice-ERP agents) and batched vision/audit inference by night on the same GPU, targeting >80% utilization of every GPU-hour. Who has tried mixed interactive+batch scheduling on a single Instinct card — scheduler pain, KV-cache thrash, anything we should fear?

3. Teacher-on-MI300X → student-at-the-edge.
We distill defect-detection and vibration/acoustic anomaly models on 8×MI300X bursts, then ship quantized students to edge devices on the plant floor. What accuracy retention are people actually achieving at edge-deployable sizes on industrial (not benchmark) data?

4. The device-side counterpoint — where should intelligence live?
Our consumer sibling, AiVibeClaw, revives aging Android devices with an on-device AI companion — the opposite philosophy: zero cloud, aging silicon, sustainability-first. As AMD pushes client-side AI (Ryzen AI NPUs), I’d love the community’s view: for always-on assistive agents, does the future split cloud-heavy (factory-style) vs. device-only (Claw-style), or converge into hybrid routing — and what’s the honest local-model floor (params/quantization) below which an on-device companion stops being useful?

WHAT I’LL GIVE BACK
We’re committing to publish our full ROCm migration + MI300X serving benchmarks (vLLM throughput, quantized 70B single-GPU results, LoRA-on-ROCm recipes, distillation retention curves) in this thread as we produce them — reproducible, warts included.

If our “three shifts at one wage” math is wrong, I’d rather hear it from this community first. Where does it break?

#AiAmbA #AiVibeClaw #AiVedhA #MI300X rocm vllm #EdgeAI #AIFactory #Industry40 #AMDInstinct