I’m the founder of AiVibe — bootstrapped, building an “AI Factory” platform called AiAmbA for small and mid-sized manufacturers, and currently onboarding onto the AMD Developer Cloud (MI300X). This is a genuine engineering discussion, not a product pitch — I want this community to attack our assumptions before physics does.
THE PREMISE WE’RE BUILDING ON
Most of the world’s factories have 2–50 machines and will never afford hyperscaler-style automation. Our bet: enrich 100+ categories of existing industrial devices through their I/O pins into “AI-brained” machines, run perception and reasoning on cloud GPU (MI300X) + on-device edge inference, and let the owner operate the plant in natural language. The economic target that makes or breaks it: one AI agent covering three shifts at roughly the cost of a single operator’s eight-hour wage.
WHERE I’D GENUINELY VALUE THIS COMMUNITY’S JUDGMENT
1. Single-GPU 70B serving as the cost anchor.
The 192 GB HBM3 on MI300X lets us serve quantized 70B-class open models (FP8/AWQ, vLLM on ROCm) on ONE GPU — no tensor parallelism, no interconnect tax. For those running vLLM on MI300X in production: what real tokens/sec and batch-depth are you sustaining, and where does quantization start costing you factory-relevant accuracy (structured tool-calls, not prose)?
2. Day/night utilization.
Our plan runs interactive voice-agent serving by day (our AiVedhA voice-ERP agents) and batched vision/audit inference by night on the same GPU, targeting >80% utilization of every GPU-hour. Who has tried mixed interactive+batch scheduling on a single Instinct card — scheduler pain, KV-cache thrash, anything we should fear?
3. Teacher-on-MI300X → student-at-the-edge.
We distill defect-detection and vibration/acoustic anomaly models on 8×MI300X bursts, then ship quantized students to edge devices on the plant floor. What accuracy retention are people actually achieving at edge-deployable sizes on industrial (not benchmark) data?
4. The device-side counterpoint — where should intelligence live?
Our consumer sibling, AiVibeClaw, revives aging Android devices with an on-device AI companion — the opposite philosophy: zero cloud, aging silicon, sustainability-first. As AMD pushes client-side AI (Ryzen AI NPUs), I’d love the community’s view: for always-on assistive agents, does the future split cloud-heavy (factory-style) vs. device-only (Claw-style), or converge into hybrid routing — and what’s the honest local-model floor (params/quantization) below which an on-device companion stops being useful?
WHAT I’LL GIVE BACK
We’re committing to publish our full ROCm migration + MI300X serving benchmarks (vLLM throughput, quantized 70B single-GPU results, LoRA-on-ROCm recipes, distillation retention curves) in this thread as we produce them — reproducible, warts included.
If our “three shifts at one wage” math is wrong, I’d rather hear it from this community first. Where does it break?
#AiAmbA #AiVibeClaw #AiVedhA #MI300X rocm vllm #EdgeAI #AIFactory #Industry40 #AMDInstinct