NVIDIA Nemotron 3.5 Bolt E4B is a small conversational model aimed at local agents. The “4GB” in its release name is a target memory envelope for particular workloads. Its official Q4_K_M GGUF is a 6.18 GB download, and an optional DFlash drafter adds another file. Neither number alone determines the memory required by your application.
NVIDIA’s release announcement positions it for voice and device assistants, interactive characters and embedded interfaces. This is the Gemma-4-E4B-derived model, not the larger Bolt 74B-A9B checkpoint. EyesTech’s existing Bolt 74B memory guide covers that separate distribution; its sizing should not be transferred to E4B.
Start with the artifacts you actually plan to load
The official file metadata gives the target GGUF as 6,180,446,208 bytes. The companion GGUF drafter repository lists its BF16 file at 736,812,096 bytes. The following values are calculated from those artifact sizes, not measured allocations.
| Artifact | Decimal GB | GiB (binary) |
|---|---|---|
| Target GGUF | 6.180 | 5.756 |
| Optional BF16 drafter | 0.737 | 0.686 |
| Combined file total | 6.917 | 6.442 |
The combined total is about 6.92 decimal GB, or 6.44 GiB. Keep sufficient disk space for the selected files and auxiliary assets. Do not treat this storage sum as a GPU-memory requirement: the runtime’s tensor placement, cache, buffers and offloading determine the actual allocation.
For a reproducible record, distinguish downloaded bytes, process memory, GPU allocation and whole-system memory. A claim that a workload fits needs the context length, active request count, cache precision and offloading settings alongside the peak. Otherwise, two apparently contradictory memory figures may simply be describing different configurations.
Use the official GGUF serving path first
The model card identifies 8B total parameters, 4B effective, and a 64K context for this distribution. Its quick start uses the CUDA llama.cpp image ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459, the file Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf, the served alias nemotron-bolt and --jinja.
Begin with the card’s “Without drafter” path. Record that baseline before introducing speculative decoding. If a first launch fails, keep the complete log: a missing architecture, an unsupported chat template and an allocation failure call for different changes. Reducing context cannot fix an unsupported model format.
For a first connectivity check, use a short synthetic request and a modest output limit. Confirm that the endpoint returns the requested served alias, that the chat template is active and that the application separates reasoning from the user-visible answer. We have not executed this recipe or measured E4B on local hardware.
The model card also provides a DFlash path with seven draft tokens. Treat that as an optional second configuration, with the correctly matched drafter. Its additional work and memory must be included when comparing performance. A faster decoding result is useful only if the complete request or agent task improves under the intended concurrency.
Keep text serving separate from a voice pipeline
The official GGUF card lists text input and text output. A local voice assistant still needs an actual route for receiving speech and producing audible output; this guide does not claim that downloading the text GGUF creates those components.
NVIDIA’s technical report describes a separate BF16 NIM package with text, image and audio input. It uses its own parsers and deployment profile. Modalities and performance from that package should not be promised for a different GGUF configuration without checking the selected runtime.
This distinction is especially useful when evaluating device assistants. Record the full path from the microphone to the final spoken reply. Include transcription or audio processing, model prefill, generation, tool execution and speech output wherever they actually occur. Token throughput covers only part of the user’s wait.
Evaluate tool use with a small controlled application
Before connecting real device actions, create a synthetic tool that records requested operations without executing them. Supply a small inventory with a missing prerequisite and ask the agent to perform a task that depends on it. Check whether it notices the prerequisite, uses valid arguments and accurately describes the tool result.
Then introduce a later correction and a delayed result. Retain every message, tool call and returned payload so a failure can be traced to the model or the application. These are proposed evaluation cases, not tests performed by EyesTech.
The useful buying or deployment decision is whether the chosen runtime meets your memory, latency and task-success requirements together. The “4GB” name is a reason to investigate a smaller local model; a recorded peak and completed-task evaluation on the actual configuration are what establish fit.
Reporting note: Prepared with AI assistance from primary documentation. Artifact-size arithmetic is calculated; proposed tests and serving steps were not executed. No independent performance result is claimed. The featured image is an original AI-generated conceptual illustration.
