DenseMem v0.2.0 — 256x KV cache compression running 72B models on a gaming PC

DenseMem v0.2.0 — 256x KV cache compression running 72B models on a gaming PC

I’m a disabled Navy veteran and solo developer. No lab, no budget, no $32k workstation. So I built a compression layer instead.

What is Dense RAM / FoldedMemory?

PCA/SVD compression on transformer KV cache activations across a two-tier VRAM/RAM hierarchy. Hot activations stay in VRAM uncompressed. Cool activations — scored by access recency and activation magnitude — get folded into system RAM at up to 256x compression with high fidelity.

The result:

  • 72B models running on consumer hardware

  • $32,000 hardware requirement reduced to $1.88 effective cost

  • ECC-safe, fidelity-verified compression

  • Designed for AMD Threadripper PRO WRX80 + 512GB DDR4 ECC across all 8 channels

The thesis: density over speed. More addressable cache pool beats faster cache for self-improving AI workloads.

What it powers:

Harvester — a self-improving AI system built on Kolb’s Experiential Learning Cycle. Reflection Engine, Growth Engine, Myelin Sheath orchestration layer, sleep/consolidation cycle. Patent pending. Achieved a perfect SAE score on the Google DeepMind Kaggle Metacognition track. Third-party reviewers independently confirmed the architecture.

Also powering NOLB (No One Left Behind) — a veteran disability claims processing system. A family member died before receiving earned VA benefits. That’s why it exists.

Happy to go deep on the compression logic, eviction policy, or architecture. Built on AMD. Built for AMD.

1 Like

This is an interesting approach to reducing KV cache memory pressure, especially for developers working with limited GPU resources. Using recency and activation magnitude to decide what stays hot in VRAM is a practical form of tiered memory management, while PCA/SVD provides a tunable trade-off between compression ratio and reconstruction fidelity.