Hi AMD community,
I want to share something we proved tonight that I think matters
for the R9700 ecosystem story.
We built Amalgis — a sovereign AI inference engine written entirely
in C and GLSL compute shaders. No PyTorch. No CUDA. No ROCm.
Pure Vulkan 1.1.
Tonight on a single Radeon AI PRO R9700 (RDNA4, 32 GB VRAM):
- 1,724 tok/s throughput at batch=32 (Mistral 7B FP16)
- 100% GPU utilization, flat — no CPU stalls, no idle
- 82W power draw — 27% of 300W TDP
- 46°C — no throttling
- VRAM ceiling at 99% (31.7 GB / 31.9 GB confirmed)
- All 32 concurrent sequences correct
- git tag: vulkan-7b-batch32-proven
The core innovation: custom parallel batch attention kernel using
k_stack/v_stack flat KV buffers. 3 GPU dispatches total per layer
with Y-dimension = batch slot. Previously required 3 × batch_size
dispatches sequentially. 9 custom GLSL compute shaders written
from scratch in C.
Throughput scaling curve:
batch=1: 17 tok/s wall → 17 tok/s throughput
batch=4: 15 tok/s wall → 60 tok/s throughput
batch=8: 26 tok/s wall → 208 tok/s throughput
batch=16: 39 tok/s wall → 619 tok/s throughput
batch=32: 54 tok/s wall → 1,724 tok/s throughput ← VRAM ceiling
GPU telemetry at batch=32 (nvtop):
GPU utilization: 100% flat
VRAM: 31.7 GB / 31.9 GB (99%)
Power: 82W / 300W (27% TDP)
Temp: 46°C
This is not a ROCm benchmark. This is a custom Vulkan 1.1 compute
kernel that runs on any Vulkan-capable GPU — AMD, NVIDIA, Intel Arc,
Apple Silicon (MoltenVK). The R9700 is our production hardware.
The engine includes:
- Custom FP16 QKV matmul (RDNA4 wave32 optimized)
- RoPE with NTK-aware theta scaling
- Parallel batch attention (k_stack/v_stack, Y=slot)
- SwiGLU FFN
- Hybrid RAG pipeline (cosine + IDF, inline, same GPU)
- Per-tenant isolated document store
- Air-gap capable, no cloud dependency
We are a DARPA SBIR applicant (FALCON DP2) building sovereign AI
infrastructure for enterprise and defense.
[attaching nvtop screenshot — 100% GPU, 99% VRAM, 82W, 46°C]
Happy to answer questions on kernel architecture, Vulkan dispatch
strategy, or GLSL shader design.
— Steven Hasson
Founder & CEO, Omniapan AI Inc.
DARPA FALCON DP2 applicant
