# First production Vulkan LLM batch decode kernel on Radeon AI PRO R9700 — 1,724 tok/s, no CUDA, no ROCm

**URL:** <https://devcommunity.amd.com/t/first-production-vulkan-llm-batch-decode-kernel-on-radeon-ai-pro-r9700-1-724-tok-s-no-cuda-no-rocm/772>\
**Category:** Artificial Intelligence (AI)\
**Tags:** ai, gpu\
**Created:** [August 3, 2026, 6:17am UTC](https://devcommunity.amd.com/t/first-production-vulkan-llm-batch-decode-kernel-on-radeon-ai-pro-r9700-1-724-tok-s-no-cuda-no-rocm/772 "2026-08-03T06:17:22Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Steven\_Hasson](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@Steven\_Hasson](https://devcommunity.amd.com/u/Steven_Hasson)\
**Post date:** [August 3, 2026, 6:17am UTC](https://devcommunity.amd.com/t/first-production-vulkan-llm-batch-decode-kernel-on-radeon-ai-pro-r9700-1-724-tok-s-no-cuda-no-rocm/772/1 "2026-08-03T06:17:22Z")

</div>

Hi AMD community,

I want to share something we proved tonight that I think matters  
for the R9700 ecosystem story.

We built Amalgis — a sovereign AI inference engine written entirely  
in C and GLSL compute shaders. No PyTorch. No CUDA. No ROCm.  
Pure Vulkan 1.1.

Tonight on a single Radeon AI PRO R9700 (RDNA4, 32 GB VRAM):

- 1,724 tok/s throughput at batch=32 (Mistral 7B FP16)
- 100% GPU utilization, flat — no CPU stalls, no idle
- 82W power draw — 27% of 300W TDP
- 46°C — no throttling
- VRAM ceiling at 99% (31.7 GB / 31.9 GB confirmed)
- All 32 concurrent sequences correct
- git tag: vulkan-7b-batch32-proven

The core innovation: custom parallel batch attention kernel using  
k\_stack/v\_stack flat KV buffers. 3 GPU dispatches total per layer  
with Y-dimension = batch slot. Previously required 3 × batch\_size  
dispatches sequentially. 9 custom GLSL compute shaders written  
from scratch in C.

Throughput scaling curve:  
batch=1: 17 tok/s wall → 17 tok/s throughput  
batch=4: 15 tok/s wall → 60 tok/s throughput  
batch=8: 26 tok/s wall → 208 tok/s throughput  
batch=16: 39 tok/s wall → 619 tok/s throughput  
batch=32: 54 tok/s wall → 1,724 tok/s throughput ← VRAM ceiling

GPU telemetry at batch=32 (nvtop):  
GPU utilization: 100% flat  
VRAM: 31.7 GB / 31.9 GB (99%)  
Power: 82W / 300W (27% TDP)  
Temp: 46°C

This is not a ROCm benchmark. This is a custom Vulkan 1.1 compute  
kernel that runs on any Vulkan-capable GPU — AMD, NVIDIA, Intel Arc,  
Apple Silicon (MoltenVK). The R9700 is our production hardware.

The engine includes:

- Custom FP16 QKV matmul (RDNA4 wave32 optimized)
- RoPE with NTK-aware theta scaling
- Parallel batch attention (k\_stack/v\_stack, Y=slot)
- SwiGLU FFN
- Hybrid RAG pipeline (cosine + IDF, inline, same GPU)
- Per-tenant isolated document store
- Air-gap capable, no cloud dependency

We are a DARPA SBIR applicant (FALCON DP2) building sovereign AI  
infrastructure for enterprise and defense.

[attaching nvtop screenshot — 100% GPU, 99% VRAM, 82W, 46°C]

Happy to answer questions on kernel architecture, Vulkan dispatch  
strategy, or GLSL shader design.

— Steven Hasson  
Founder & CEO, Omniapan AI Inc.  
DARPA FALCON DP2 applicant

 ![amalgis-Screenshot 2026-08-02 at 1.34.04 PM](https://global.discourse-cdn.com/amd/original/2X/5/5f3bb2fb217df37f8f5e32d9aa591c513e339f3f.png)