This post is from a low-reputation account and contains an unverified outbound link. Be cautious before clicking external links.

NVIDIA Just Squeezed a Smart Conversational AI Into 4GB of VRAM

abir70(25)

Published in

#ai

Words

462

Reading

3 min

20h

Listen to this post

~3 min

Your robot, your game character, or your home voice assistant may never need to phone home to a cloud server again.

Yesterday, NVIDIA released Nemotron 3.5 Bolt E4B 4GB — an open small language model built to run conversational agents entirely on local hardware. Eight billion total parameters, four billion effective, with a 64K-token context window. In NVIDIA's own tests, the Q4_K_M build used between 3.1 and 4.3 GB of VRAM on RTX cards, leaving room on the same GPU for the speech and vision models a real assistant needs.

The model is post-trained from Google's Gemma 4 E4B using reinforcement learning only. NVIDIA is aiming it at three workloads: device and voice assistants, interactive game characters, and embedded or industrial agents. One of the launch demos is a Reachy Mini robot running on a Jetson Orin Nano with just 8GB of memory — Cosmos 3 Edge handles vision while Bolt handles speech replies and tool calls. NVIDIA also calls Bolt the third-generation small model for its ACE platform, the tech behind lifelike NPCs in games.

The numbers come with a caveat, as they always should. NVIDIA's measured benchmarks put Bolt E4B on top in 15 of 21 tests against Gemma 4 E4B, Qwen3.5-4B, and Nemotron 3 Nano 4B — the best result was IHEval at 84.8, against 67.0 and 60.6 for the other two. But on Tau2-Bench it scored 55.1, roughly 20 points behind Qwen3.5-4B. Tool-calling in messy, multi-step situations is clearly still its weak spot, and these are NVIDIA's own measurements, not independent ones.

Still, the timing matters. Bolt E4B is the final piece of NVIDIA's Nemotron 3.5 lineup announced on October 15: Super VL, the 120-billion-parameter flagship for enterprise agents, and Bolt 74B-A9B, a 74B mixture-of-experts model squeezed into 24GB through native 2-bit quantization — both available since day one. The 4GB edge model was the missing rung, and it landed exactly when NVIDIA said it would, on October 23.

Zoom out and the pattern is hard to miss. October has become an open-weight month: Reflection's 501-billion-parameter Beam, Mistral's trillion-parameter Large 4 preview, Microsoft's on-device MAI-Code-1.1-Flash, NVIDIA's own 74B model. After a year of supply-chain anxiety in the model world, companies are competing on who can hand you the weights — not just the API.

Which leaves a fun question for the rest of us. If a genuinely capable assistant can live on a GPU you already own, with no per-token bill and no data leaving your room — do you still rent answers from the cloud? I'd genuinely like to hear where you land on this.