Microsoft-Decision-1: 9B Cloud Model in a 35ms Local World

Microsoft has entered the specialized decision-engine arena with Microsoft-Decision-1, a dedicated model engineered not to compose text, but to score discrete choices. Unveiled by Microsoft CEO Satya Nadella, the model evaluates complex contexts up to 32,768 tokens and returns calibrated probability distributions over predefined options in a single forward pass.

Introducing Microsoft-Decision-1, our new model for fast decision-making.

It delivers top performance on structured decision tasks, outperforming both LLMs and other decision models in latency and quality.

We’re already testing it across Microsoft for everything from incident response and quality control to scientific discovery.

— Satya Nadella (@satyanadella) October 10, 2026

Available immediately through Microsoft Foundry and slated for OpenRouter, Decision-1 carries an aggressive pricing structure of $0.042 per million input tokens, with output tokens provided at no charge. Microsoft reports an 85-millisecond median (P50) latency across 36 evaluation benchmarks, touting a 35-fold speed advantage over GPT-6 Sol.

Yet behind the headline speedup lies an architectural and competitive tension that standard coverage has largely overlooked: Microsoft has deployed a 9-billion parameter model hosted across remote hyperscaler regions, stepping into an ecosystem where startup pioneers like TypeSafe AI’s Jev and open-source models like Laya have already established sub-35ms structured decision engines running entirely on-device.

The Ghost in the Benchmark: Why Jev Was Sidelined

When Microsoft released its 36-dataset benchmark suite spanning nearly 150,000 blind questions, the comparative latency table highlighted dramatic margins against general-purpose generative LLMs:

• Microsoft-Decision-1: 85 ms (P50 median)
• H2O-Lightning-4B v1.1: 210 ms (2.5× slower)
• JevBench (adjusted): 240 ms
• GPT-6 Luna Decisions: 300 ms (3.5× slower)
• GPT-6 Sol: 3,010 ms (35× slower)

Comparing a specialized single-pass classification head against GPT-6 Sol—an autoregressive, 100B+ parameter frontier model generating tokens sequentially—is an asymmetric comparison. In production systems, engineering teams do not deploy multi-thousand-millisecond reasoning giants to execute binary routing gates or triage labels.

The benchmark comparison that developers on X and Hacker News immediately looked for was a head-to-head evaluation against TypeSafe AI’s Jev.

Over the past month, Google Trends telemetry showed search volume for typesafe jev and jev model ai exploding to peak category breakout levels. Jev pioneered commercial sub-35ms decision scoring, establishing the API protocol that catalyzed the entire “System-1” model movement. When teams upgrading to frontier reasoning pipelines hit massive cost walls, they turned to the Jev + Opus 5.5 gatekeeper architecture to filter routine prompts before invoking costly generation cycles.

Instead of evaluating Decision-1 directly against live, dedicated Jev API endpoints, Microsoft’s published documentation relied on an adjusted JevBench latency metric pegged at 240 ms. In response, competitor H2O.ai publicly contested the figures, noting that its self-hosted H2O-Lightning-4B executes at 29 ms in dedicated on-prem environments rather than the 210 ms recorded in Microsoft’s remote cloud evaluation environment.

Vendor-run benchmarks are inherently tied to evaluation network topology. By framing its primary marketing delta against autoregressive giants, Microsoft secured its “35x faster” soundbite while leaving direct edge parity unaddressed.

Parameter Inflation: 9B in the Cloud vs. 0.4B on Silicon

The most consequential engineering divergence in Microsoft-Decision-1 is its physical weight: 9 billion parameters, post-trained on Alibaba’s open-weight Qwen3.5-9B backbone.

To understand why this choice matters, consider where the decision-model sector originated. When Convai Innovations released Laya on Hugging Face, it engineered the engine around a 421M-parameter ModernBERT backbone. Shortly thereafter, the community ported that logic into the Unsloth Local Decision API, allowing engineering teams to run full Jev-compatible wire protocols on consumer Apple Silicon and desktop GPUs in 33 milliseconds offline.

Here is how the competing models compare across architecture, hosting tier, and operational parameters:

The Network Round-Trip Reality

An 85-millisecond P50 inference latency inside an Azure datacenter does not equal an 85-millisecond latency on a production server.

When an agentic loop running in an external cloud (AWS, GCP, or on-prem) calls Microsoft Foundry over a wide-area network (WAN), the physics of TCP handshakes, TLS negotiation, and packet routing apply:

In cross-cloud or transatlantic production paths, network RTT typically adds 45 to 80 milliseconds. As a result, the end-to-end wall-clock latency for Decision-1 reaches 130 ms to 165 ms.

For asynchronous batch processing—such as grading overnight conversational logs or batching 10,000 bug tickets—this delay is trivial. But for synchronous, real-time agent loops where an orchestrator makes 15 discrete branching choices per user turn, a cumulative 2.2-second wall-clock delay (including up to 1.2 seconds in network transit alone) is significant. In contrast, a 421M local model like Laya residing in local VRAM incurs zero network transit, processing decisions in a deterministic 33 ms.

What Microsoft gains by scaling to 9B is semantic capacity: Decision-1 can digest dense 32k-token prompts containing technical manuals, complex enterprise policies, or multi-field rubrics that would saturate the limited attention span of a 400M-parameter BERT model.

The Realpolitik of Qwen3.5: Why Alibaba Won Microsoft’s Stack

One of the most revealing technical details in the Decision-1 release is its provenance. Despite Microsoft’s multibillion-dollar alliance with OpenAI and its internal Phi and MAI (Microsoft AI) research groups, Decision-1 was post-trained on Alibaba’s open-weight Qwen3.5-9B.

While Microsoft noted that it plans to rebase future iterations on MAI models and OpenAI architectures, the decision to launch on Qwen reflects pragmatic engineering: 1. Dense Representation Quality: Qwen3.5’s base pre-training offers exceptionally clean token representations and instruction adherence in the sub-10B class. 2. Post-Training Adaptability: Repurposing a decoder backbone into a non-autoregressive decision head requires a model whose latent states preserve fine-grained semantic boundaries across extended context lengths. 3. Speed to Market: In a race where Cloudflare deployed Clef on Qwen backbones and the open-source community rallied behind Laya, building from scratch on internal experimental weights would have ceded another quarter of enterprise market share.

The reality is unmistakable: when an American hyperscaler needed to establish an enterprise decision platform, the most capable open foundation was engineered in Hangzhou.

Single-Pass Mechanics: How Decision-1 Scores Without Decoding

Traditional generative models require an autoregressive loop: they sample token t₁, append it to context, run another forward pass to sample t₂, and continue until emitting a closing JSON bracket. If the model hallucinates formatting or outputs preamble text (“Here is your decision:”), downstream parsing breaks.

Decision-1 bypasses autoregression entirely. The input payload passes the context and a bounded candidate array into the 9B transformer. A specialized classification head reads the sequence’s final hidden states and projects logits directly across the specified options:

Because output tokens are not generated sequentially, Microsoft does not charge for generation. The $0.042 per million input tokens pricing reflects pure prefill compute.

Adversarial Perturbation Resilience

A persistent vulnerability of using standard LLMs as judges or routers is positional and phrasing fragility: * Moving the correct choice from Option A to Option C can shift model selection by up to 18% in standard instruction models. * Minor semantic paraphrasing often flips routing decisions.

In Microsoft’s adversarial resilience evaluations across eight perturbation classes (including prompt rephrasing, option shuffling, and irrelevant distractor insertion), Decision-1 maintained a 1.3% average decision-flip rate. When option descriptions were simply shuffled or paraphrased, the model recorded 0.0% decision flips.

This stability is critical for regulated enterprise environments—such as Azure incident response or Xbox trust and safety—where non-deterministic routing can trigger audit failures.

Architectural Blueprint: The Two-Tier Agent Pattern

The launch of Decision-1 solidifies an emerging consensus in production engineering: the monolithic “one LLM does everything” design is obsolete. Production architectures are standardizing on a Discriminator-Generator Hierarchy:

In this structure: 1. Tier 0 (The Gatekeeper): Runs on Decision-1, Laya, or Clef. It handles policy compliance, user intent routing, tool selection, and state validation in sub-100ms. 2. Tier 1 (The Synthesizer): Frontier generative models are invoked only when open-ended drafting, multimodal creation, or deep symbolic synthesis is strictly necessary.

By filtering 70% to 85% of intermediate decision steps through a $0.042/1M input scoring layer, enterprise agent loops avoid thousands of dollars in generative token waste while shaving seconds off cumulative execution traces.

Deployment Rubric: When to Adopt and When to Run Local

For engineering teams assessing Microsoft-Decision-1, the selection criteria depend on three concrete operating constraints:

1. Context Density vs. Network Topology: If your routing logic evaluates concise strings (<1,000 tokens) in latency-critical desktop or on-device loops, self-hosted alternatives like Laya via Unsloth remain superior due to zero network overhead and local privacy boundaries. If your decisions require parsing 20-page legal contracts, enterprise telemetry logs, or extensive system prompt rubrics, Decision-1’s 32k context window and 9B depth provide necessary capacity.
2. Egress and Lock-in: While input pricing is modest, routing your operational control layer through Microsoft Foundry introduces architectural coupling to Azure infrastructure. Teams operating in multi-cloud environments must weigh managed convenience against the sovereign flexibility of open-weight alternatives.
3. Calibrating the Abstention Threshold: Decision-1 includes an explicit abstention score when context is ambiguous. Before granting the model autonomous write permissions in production pipelines, teams should validate confidence calibration against proprietary historical data rather than vendor benchmark indices.

Microsoft-Decision-1 validates the thesis that non-autoregressive decision models are the foundation of reliable agentic infrastructure. The challenge for developers is no longer deciding whether to adopt a dedicated decision model, but deciding whether to host it at the local edge or pay the hyperscaler network tax.