I’m building ASEMA — an inference architecture for running 763B+ parameter sparse MoE models on 32GB RAM + 8GB VRAM
Hi AMD community,
I’m a student developer from India, and I’ve been working on something that started as a simple question:
What happens if you try to run an extremely large sparse Mixture-of-Experts model on hardware that is nowhere near large enough to hold the entire model in RAM or VRAM?
Instead of shrinking the model, pruning experts, distilling it, or sending inference to a cloud API, I decided to attack the memory and execution problem itself.
The result is ASEMA:
Adaptive Sparse-Expert Memory Architecture.
The core idea is to treat storage, system memory, GPU memory, and compute as a coordinated hierarchy.
NVMe storage
↓
Expert paging / asynchronous I/O
↓
RAM resident cache
↓
GPU resident working set
↓
MoE routing
↓
GPU expert execution
↓
Token generation
My current experimental platform is:
• AMD Radeon RX 580 8GB
• AMD Ryzen 7 5700X
• 32GB system RAM
• Windows 11
• Dual NVMe storage
The model I have been working with is DeepSeek-V4.1-Flash, a very large sparse MoE checkpoint with a total footprint vastly beyond the memory available on this machine.
The architecture I have been implementing includes:
• Real model tensor loading
• Sparse MoE routing
• Top-k expert selection
• FP4 expert execution
• Expert caching
• Asynchronous storage I/O
• Prefetching
• RAM/VRAM residency management
• CPU/GPU execution
• Persistent GPU resources
• KV-cache infrastructure
• Telemetry and profiling
• Memory-bounded execution
• Correctness verification against reference calculations
One of the most interesting constraints is that I am deliberately NOT trying to load the entire model into RAM or VRAM.
The goal is to keep the complete model represented on storage while dynamically bringing the tensors required for the current computation into the active memory hierarchy.
This turns the problem from:
“How do I fit a gigantic model into memory?”
into:
“How do I design an inference runtime that intelligently moves only what computation needs, exactly when it is needed?”
That is the problem ASEMA is trying to solve.
I have also been experimenting with custom GPU execution on the RX 580 rather than treating the GPU simply as a conventional inference backend.
My previous experiments demonstrated GPU-accelerated FP4 expert execution and storage-to-GPU execution pipelines under strict memory constraints.
The next stage is much more ambitious.
I want to make ASEMA model-independent and investigate whether the same architecture can be extended from the current 763B-class experiment toward trillion-parameter and multi-trillion-parameter sparse MoE models.
I recently joined the AMD AI Developer Program, and I would especially like feedback from the AMD/ROCm community on the architecture.
In particular, I would love feedback on:
-
Which parts of the storage → RAM → VRAM pipeline would you redesign for AMD hardware?
-
What would be the best ROCm/HIP architecture for persistent expert execution and asynchronous expert movement?
-
Which AMD profiling tools should I use to identify the largest bottlenecks?
-
Are there AMD-specific approaches for improving sparse MoE expert execution that I should investigate?
-
Would this architecture be interesting as an open-source AMD/ROCm project?
I’m planning to open-source ASEMA once the current implementation reaches a reproducible state with proper benchmarks and documentation.
I’m not looking for hype — I’m looking for technically serious feedback.
If anyone here works with ROCm, GPU kernels, inference systems, sparse MoE models, or large-model serving, I would genuinely appreciate your thoughts.
Project: ASEMA
Focus: Sparse MoE inference / memory hierarchy / expert paging / GPU acceleration
Hardware: Ryzen 7 5700X + Radeon RX 580 8GB + 32GB RAM
This started as an experiment on a consumer GPU.
I’m now trying to turn it into a real inference architecture.