🤯 This shouldn't work 👉 DeepSeek V4 Flash on 7.7GB RAM. No GPU, and instead using an increasingly popular and more affordable technique.
DeepSeek V4 Flash is a 284.33B-parameter MoE.
The GGUF used here is 78.62 GiB.
The machine?
💾 8 GB RAM
🎮 ZERO GPU / CUDA
🧠 CPU-only
💿 NVMe SSD
📦 78.62 GiB model
🧮 284.33B parameters
They got the model
to execute.
Even crazier ⚡ Controlled cold first-token diagnostic: 5.33 sec 🧠 Maximum RSS: ~5.9 GiB 🔄 Process swaps: 0 ⚠️ No tps reported How? The whole 79GB model never goes into RAM.
The GGUF stays memory-mapped on the NVMe SSD, and Linux demand-pages the pieces of the model into RAM as they’re needed.
💬 NVMe becomes backing storage for model weights → RAM becomes a moving working set. This is NOT fast.
The developer explicitly says they are not claiming competitive sustained decode speeds, and the 5.33-second result is a controlled first-token diagnostic, not normal chat latency.
🎯 So the fact that the model doesn't fit in RAM is more of a performance problem rather than an NVRAM barrier.
If smart paging + MoE expert loading + faster NVMe keeps improving, the SSD may become another tier in the Local AI memory hierarchy:
VRAM → RAM → NVMe