NVIDIA Nemotron 3.5 Super VL supports native multi-token prediction, while DGX Spark also has a separate LiLiCorr speculative drafter. Both propose tokens for the target model to verify. They are different serving configurations, and their speed depends on hardware, draft acceptance, concurrency and the amount of time spent outside decoding.
The NVFP4 model card describes a 120B-total, 12B-active hybrid vision-language model with up to a 1M-token context. It accepts text, images and video and produces text. Neither the active-parameter count nor the context maximum is a complete hardware requirement.
Native MTP and LiLiCorr load different components
With native MTP, the trained prediction head is part of the target model. NVIDIA’s vLLM and SGLang MTP recipes do not require a separate drafter download. The engine proposes a block of future tokens and verifies it through the target, advancing through accepted candidates.
The separate LiLiCorr checkpoint is designed for DGX Spark and low-concurrency data center serving. It must be loaded alongside the target. Its documented architecture includes 1.13B trained parameters, including a correlation head, plus a 0.54B NVFP4 copy of the target’s output head.
NVIDIA’s technical explanation says LiLiCorr scores compatibility across parallel draft candidates. The goal is to form a coherent draft for verification, instead of treating the predictions at successive positions as unrelated choices. NVIDIA recommends it for low-latency Super VL inference on DGX Spark and reports up to 2.3× faster token generation than decoding without speculation.
| Serving route | Draft component | What must be checked |
|---|---|---|
| Native MTP | Prediction head within target model | Runtime support, configured draft length and measured acceptance. |
| LiLiCorr | Separate checkpoint loaded beside target | Matched target/drafter, method, memory and concurrency. |
| Spark quick start | LiLiCorr via dflash; 7 draft tokens | One-sequence recipe; preserve its hardware-specific settings. |
More proposed tokens do not guarantee more progress. Rejected candidates still consume draft and verification work. The useful quantity is how much accepted output arrives per unit of total time under the intended workload.
Select the complete recipe for the machine
The model card’s Spark quick start uses LiLiCorr through vLLM’s dflash method, seven speculative tokens and one concurrent sequence. Other documented hardware paths use native MTP with their own draft lengths. Do not combine a method from one recipe with the memory and concurrency assumptions of another.
For an initial deployment, record the target revision, drafter revision if present, container image, GPU, configured context and maximum sequences. Retain the full launch command. Check startup logs to confirm the requested speculative method was selected and the expected extra checkpoint loaded when using LiLiCorr.
The serving examples also configure model-specific reasoning and tool-call parsers. A healthy HTTP endpoint establishes reachability. It does not demonstrate that reasoning fields or tool calls are parsed correctly by the connected agent.
The checkpoint’s OpenMDW-1.1 license is separate from hosted endpoint terms. Check the exact distribution being deployed rather than assigning the same license, precision or hardware profile to the entire Nemotron family.
Throughput tables need their cache and concurrency
NVIDIA’s Dynamo recipe results list an agentic trace target with median input length of 222,076 tokens, median output length of 188 and a 99% KV-cache hit rate. That cache condition materially changes how much of the input must be processed again.
As a rough arithmetic illustration, 1% of 222,076 is about 2,221 tokens. This is not a reconstructed request trace: cache accounting, distributions and scheduling prevent an exact latency estimate from that multiplication. It does show why a fresh 222K-token document should not inherit the latency expectation of a heavily reused conversation.
The same results distinguish system output tokens/s/GPU from median output tokens/s/user. For the B200 vLLM aggregated recipe at concurrency 128, the table reports 1,378.9 system output tokens/s/GPU and 68.86 median output tokens/s/user. Those are different denominators. Dividing one by the other does not recover a reliable user count or per-request service guarantee.
The benchmark procedure replays recorded SWE-bench Pro trajectories through AIPerf. It explicitly measures serving throughput and latency, rather than rerunning repository tasks to measure issue-resolution accuracy. Keep that distinction when using a serving chart to make a coding-agent decision.
Translate decoding speed into a task-time estimate
Suppose an application takes 100 seconds before speculation: 30 seconds decoding and 70 seconds in prefill, tool execution and other work. If decoding alone becomes 2.3 times faster, with every other component unchanged, the total becomes:
70 + 30 ÷ 2.3 = approximately 83.0 seconds.
That is about a 17% reduction in total time, or 1.20× overall speed—not a 2.3× application speedup. This is an illustrative workload, not a measurement of Nemotron. It assumes no extra scheduling cost, no changed outputs and no additional retries.
The example also explains when speculation is most useful: decoding must account for enough of the original delay. A workflow dominated by searching, builds or long uncached prompts can show a modest improvement even when its token-generation rate rises sharply.
For a useful comparison, preserve prompt and image inputs, sampling parameters, reasoning budget, tool responses, concurrency and cache state. Compare a no-speculation baseline with native MTP and LiLiCorr as separate configurations. Record total task time, time to first token, accepted-token behavior, memory use and successful task outcomes.
Repeat under both warm and cold cache conditions if the application uses both. Include queueing and retries when estimating capacity or cost per completed task. A configuration that helps one interactive user may need a different draft length or may lose its advantage as simultaneous traffic grows.
That evidence would support a deployment decision: the smallest practical serving cost at the required quality and response time. A headline token rate can identify a configuration worth evaluating, but it cannot supply the missing workload measurements.
