On June 4, 2026, Nvidia closed out its Nemotron 3 lineup with Nemotron 3 Ultra, a 550B-parameter MoE with 55B active and fully open weights. It is the most intelligent open-weight model from a US lab, but it still trails the Chinese open frontier led by Kimi K2.6.
Nvidia's pitch is not the highest benchmark scores, but the highest throughput on long-running agentic workloads. Let's walk through the recipe and poke at the inference numbers behind that claim.
Architecture
Ultra is the biggest of the three Nemotron 3 models. All three are MoEs that pair Mamba-2 layers, which run in linear time with no growing KV cache, with a thin set of attention layers for exact recall.
The one constant worth noting is sparsity: all three keep about 90% of their weights idle per token, even as total parameters grow 17x. The hybrid backbone is otherwise standard, the same linear-time-plus-attention mix Liquid, MiniMax, and Qwen use. Two blocks are more particular:
LatentMoE (paper): compresses the expert dimension about 4x, which lets Nvidia run 512 experts with 22 active per token at normal cost. Nano skips it for a conventional 128-expert, top-6 setup.
Multi-Token Prediction (MTP): two tied heads predict ahead, baking the speculative-decoding drafter into the weights. DeepSeek introduced MTP in V3, and Nvidia ties the two heads so the drafter stays aligned with the model.
Laid side by side, the family is one recipe instantiated at three different lengths:
Pre-training
Ultra is the largest model pretrained end-to-end in NVFP4, Nvidia's 4-bit float format. The recipe is Super's, scaled up: keep the most sensitive tensors in higher precision and run the rest in FP4.
What surprised me most is the token budget. Ultra trained on only 20T tokens, fewer than the 25T behind both Nano and Super. Normally the biggest model gets the most data, not the least. Nvidia cut the run short because, by its own account, it diverged twice:
Around 8T tokens, a throughput tweak that moved the output layer's gradient accumulation to BF16 quietly zeroed out the MTP heads' contribution, and the MTP loss spiked before the main loss showed anything. Reverting to FP32 fixed it.
Around 16T tokens, a second divergence Nvidia never root-caused. They annealed the learning rate early and stopped at 20T instead of the planned horizon. The report links it to dead experts and a ballooning residual stream, but only by correlation.
The curves show the call. The original run (orange) spikes near 16T, the early-annealed runs keep dropping, and the 20T version (red) is the one Nvidia shipped.
NVFP4 pretraining is still experimental at this scale. The run diverged twice; Nvidia never explained the second one, and the instability cost Ultra a fifth of its planned tokens. FP4 buys inference throughput, but it still adds real training risk.
Post-training
Post-training runs in four stages: a base model, supervised fine-tuning, reinforcement learning, then several rounds of distillation.
The first two stages are routine. SFT is multi-domain and keeps the shared-weight MTP objective. RLVR then runs a single reinforcement-learning stage with verifiable rewards across every domain at once: agentic, reasoning, chat, safety, and long context, producing one generalist checkpoint. That generalist plateaus because, with more than a dozen environments mixed together, each domain gets too little signal per batch.
To get past that, Nvidia also trains more than ten domain specialists, each a separate model tuned hard on one area: SWE, terminal use, search, safety, and so on, through its own SFT and RL. These specialists are the teachers, and the generalist is the student.
It merges them with on-policy distillation (GKD, 2023), where the student generates its own rollouts and each teacher grades them token by token. Nvidia runs this over two iterations, with a brief warm-up SFT first so the student's rollouts stay on the teachers' distribution. The first round distills the specialists into an interim model, the second reuses them plus a fresh batch to produce the final Ultra. Nvidia calls it Multi-teacher On-Policy Distillation (MOPD)1.
The contrast with DeepSeek-V4 is the interesting part. DeepSeek dropped its RL stage and replaced it with multi-teacher distillation. Nvidia keeps RL and stacks MOPD on top. DeepSeek treats distillation as a replacement for RL, Nvidia treats it as a consolidation layer that RL feeds into.
The payoff is lopsided. Recovery runs from over 170% on Terminal Bench, where the student beats its teacher, down to 17% on HLE. Execution and tool-use recover almost fully, and hard reasoning barely moves.
Nvidia is precise about why the reasoning gap stays open. Its reasoning teacher gained its edge from extra SFT and RL on a DeepSeek-V4-Pro reasoning mixture the student never saw, so the teacher's reasoning paths fall outside what the student samples. On-policy distillation only grades the student's own rollouts, so it cannot transfer a skill the student never produces. It amplifies reasoning the student can already reach, not reasoning beyond it.
The same pattern shows up elsewhere. Thinking Machines lifted a Qwen3-8B from 60 to 70 on AIME with this method, because that student could already reach AIME-level traces, and DeepSeek-V4 reasons well because its base could. My read is that the lever is the base. Ultra, undertrained by 5T tokens, does not reach far enough on the hardest problems, which is why HLE barely moves.
Benchmarks
Nvidia's own table and Artificial Analysis agree on where Ultra stands. The Nvidia table has it among the best open models on agentic and long-context work, and behind DeepSeek-V4 and Kimi on hard reasoning like HLE and GPQA.
Read down the columns, and the problem is hard to miss. Ultra holds the top score in only one of eleven benchmarks (RULER at 1M), and a Chinese model, usually Kimi, takes most of the rest. For a 550B flagship measured against models its own size and smaller, that is a thin result.
Artificial Analysis breaks it down the same way. Ultra leads on instruction-following, professional tasks, and long context, and falls behind on coding and long-horizon planning, exactly where Kimi K2.6 and GLM-5.1 keep their edge. On the Intelligence Index, it’s the top US open model, but it lands about six points behind Kimi K2.6, a real gap on a 100-point scale.
None of this is a surprise after the post-training section. The place MOPD gained the least, hard reasoning, is exactly where Ultra falls short, so the category split here just echoes the recovery numbers.
Speed is the one axis where Ultra clearly wins. A pre-release DeepInfra endpoint ran it above 300 tokens per second, against the 50 to 100 typical of comparable Chinese models. Nvidia's own "up to 6x" is shakier. It is 5.9x against Kimi but only 1.6x against Qwen, the one rival with a similarly lean active path, and it was timed on Nvidia's own stack against rivals on vLLM. As implicator.ai put it, these are "Nvidia's own numbers, against rivals Nvidia chose." The lead seems real, but the exact multiple might not hold in practice.
The bigger picture
Nvidia barely needs Ultra to win. It sells the silicon nearly every frontier model trains and runs on, so trailing Kimi costs it little. The real risk is not a better Chinese model, but a better Chinese chip. DeepSeek-V4 already trains and serves on Huawei Ascend, and as more Chinese labs move off Nvidia, the moat Ultra quietly defends erodes.
None of this takes away from the work. A stable 550B run in 4-bit, a ten-teacher distillation pipeline, and fully open weights are hard to land. Congratulations to the team that shipped it!
Resources
Technical report: Nemotron 3 Ultra: Open, Efficient MoE Hybrid Mamba-Transformer for Agentic Reasoning
Recipes & data: github.com/NVIDIA-NeMo/Nemotron
Funnily enough, Nemotron Cascade 2 already employed the acronym “MOPD” to talk about Multi-domain On-Policy Distillation instead of Multi-teacher On-Policy Distillation.












V nice write-up.
1. Why dyou think Nvidia trains Nemotron? To keep up capabilities? To offer a model that is "western frontier OSS"? [although includes distilling DeepSeek v4].
2. Why dyou think on policy student roll-outs are done rather than just direct distillation from the teacher. Does on-policy actually give better results (because it's on-policy), or, is it cheaper because you don't need to do a full decode on the teacher?
Thanks again.