This Open-Source Model Just Killed Jev: Inside Ollaya, the “Ollama for Decision AI” Crushing Cloud APIs in 8ms

The Category Shift: From Autoregressive Slop to Instinctive Decisions

When TypeSafe AI unveiled Jev in late 2026, it felt like the enterprise software industry had finally acknowledged its greatest architectural bottleneck. As we explored in our deep dive on why TypeSafe’s Jev proved we were wrong about LLMs, engineering teams had spent four years forcing conversational, autoregressive chat models into programmatic decision pipelines.

To classify an incoming webhook, triage a support ticket, or route an API payload, systems were routinely spinning up 70-billion-parameter text generators. Applications waited 2 to 4 seconds while GPUs sequentially projected formatted JSON syntax token-by-token:

{ "intent": "refund", "urgency": "high", "risk": 0.85 }

Ninety percent of the computational budget was burned generating structural formatting—quotes, whitespace, and brackets. Worse, because next-token prediction cross-entropy loss optimizes for linguistic plausibility rather than mathematical certainty, standard LLMs regularly hallucinated invalid schema keys and generated uncalibrated confidence numbers.

TypeSafe proved that non-autoregressive System 1 Decision Models—which accept arbitrary application state alongside typed schemas and project calibrated probabilities in a single forward pass—could reduce decision latency to 250 milliseconds.

Yet TypeSafe replicated the classic corporate walled garden:

1. Jev is strictly closed-source.
2. The weights are locked behind a proprietary cloud API.
3. Every sensitive support ticket, internal email, and proprietary document payload must leave the enterprise security perimeter.
4. Despite skipping token generation, network handshakes and hosted queueing pin Jev’s median latency between 236ms and 276ms.
5. Every single decision meters your cloud bill.

Enter Mert Cobanov and Ollaya.

In a single release that has set the open-source AI community ablaze, Ollaya does for decision models what Ollama famously did for generative LLMs: it packages verified open weights into a unified runtime, serves a local HTTP and MCP daemon on 127.0.0.1:11435, and executes typed decisions in 8.9 milliseconds.

Proprietary System 1 didn’t even have six months to breathe. It just got commoditized.

What Is Ollaya? The Ergonomics of Ollama, Built for Deciders

To understand why Ollaya is spreading so rapidly through engineering organizations, look at developer ergonomics.

Before Ollama arrived in 2023, running an open-weight Llama model locally was a developer headache: engineers had to juggle Python virtual environments, configure llama.cpp compile flags, manage Hugging Face tokenizers, and configure GGUF quantizations. Ollama collapsed that entire mess into two universal commands: ollama run and ollama serve.

Ollaya brings that exact developer experience to the decision model ecosystem:

# Pull and inspect installed decision models
cobanov@cobanov-air ~
> ollaya list
NAME                     ID             SIZE       MODIFIED
decider:0.8b             3b67acf91b91   1.5 GB     1 hour ago
gliclass:latest          5ee71b9d3412   1.8 GB     1 hour ago
nli:latest               8d4b54e16676   884 MB     1 hour ago
laya:latest              b87ca1631b11   11 KB      1 hour ago
laya:multilingual        ba7a334675b4   684 MB     1 hour ago
laya:en                  bf30e4654e94   854 MB     1 hour ago
laya:typed-decisions     c14e927fa102   854 MB     1 hour ago

# Run an immediate classification in your terminal
> ollaya run laya --preset triage "I was charged twice for my subscription this month and want a refund."
intent             refund                   1.00 [====================]
is_urgent          no                       0.88 [==================  ]
frustration        1.76 / 3 clearly annoyed 0.36 [=======             ]
refund_requested   yes                      0.90 [==================  ]
churn_risk         no                       0.61 [============        ]

# Spin up the background daemon
> ollaya serve
2026-09-24T12:13:08.856568Z INFO ollaya_server::http: Ollaya is running address=127.0.0.1:11435, [::1]:11435 version="0.3.1" models=/Users/cobanov/.ollaya/models

There are no streaming tokens, no markdown backticks to strip, and no temperature hallucinations. You pass state; you get typed fields, scalar scores, and mathematical probability distributions.

The Architecture: Why a Single Forward Pass Obliterates Autoregressive Loops

The fundamental difference between a conventional LLM, a cloud decision engine like Jev, and a local runtime like Ollaya comes down to memory bandwidth and network physics.

When a standard generative LLM answers a classification question, it operates inside an autoregressive loop. Because transformers generate one token at a time, each new token requires reading the entire parameter weights from High Bandwidth Memory into GPU compute registers. If an LLM needs 40 tokens to produce a structured JSON response, it performs 40 full memory transfers and 40 sequential forward passes. This is why even a small 7B model requires several seconds to produce a simple classification verdict.

TypeSafe’s Jev solved the token-generation problem by evaluating schemas in a single forward pass, but it introduced a different bottleneck: the cloud boundary. Even when inference takes 40ms on TypeSafe’s servers, the overall request incurs TLS handshakes, public internet ingress, cloud API gateway routing, multi-tenant queueing, and response egress. In real-world enterprise conditions, that round-trip overhead pushes Jev’s median latency to 236–276ms.

Ollaya eliminates both bottlenecks simultaneously. By running locally over ONNX Runtime and native CUDA (listening on 127.0.0.1:11435), Ollaya removes the network boundary entirely. The input text and schema criteria are encoded together in memory. The runtime executes a single forward pass, immediately projects the outputs onto classification heads or logit distributions, and returns the response across localhost in under 10 milliseconds.

The Models Inside: The Open-Weight Engines Powering Ollaya

It is essential to understand that Ollaya itself is the orchestration and serving framework—much like Ollama is for llama.cpp. The actual intelligence and decision accuracy come from the underlying open-weight decision models that Ollaya pulls, optimizes, and serves.

Ollaya ships with first-class support for four distinct model architectures, each tailored for specific latency, multilingual, and accuracy profiles:

1. Laya (by Convai Innovations)

As we documented when Laya hit #1 on Hugging Face, this model family is purpose-built for non-autoregressive System 1 decisions under the Apache-2.0 license:

• laya:en (854 MB, ModernBERT-large backbone, 421M params): Context length 512. Optimized strictly for English-language guardrails, triage, and rapid classification. Runs in 16.3ms on RTX 4090.
• laya:multilingual (684 MB, mmBERT-base backbone, 322M params): Context length 1024. Fine-tuned across 100+ languages, running up to ~2.2× faster on batched calls (9.1ms on RTX 4090).
• laya:typed-decisions (854 MB, ModernBERT-large backbone, 421M params): Specifically fine-tuned for typed-decisions workflows. Reaches 0.766 accuracy on the benchmark, surpassing Jev 1.13’s published 0.727.
• Routing Intelligence: When calling laya:latest without specifying a tag, Ollaya automatically routes English inputs to laya:en and non-English text to laya:multilingual.

2. Decider (by Mapika on Qwen3.5)

For workloads that demand frontier-grade semantic reasoning without sacrificing single-pass execution, Mapika developed Decider:

• Variants: Available as decider:0.8b (1.5 GB) and decider:2b (1.9 GB) built on Qwen3.5 decoder backbones.
• Mechanism: Instead of generating freeform text, the model reads candidate answers directly from option-letter logits (A, B, C, D) in a single unified forward pass.
• Benchmark Standing: Decider is currently the most accurate open decision model available in the Ollaya ecosystem, achieving 155ms on 0.8B and 190ms on 2B while handling complex multi-option nuances.

3. NLI (by Moritz Laurer)

Built by leading zero-shot NLP researcher Moritz Laurer, the nli model family leverages Natural Language Inference:

• Variants: 396M and 435M checkpoints based on fine-tuned DeBERTa-v3 architectures.
• Mechanism: Every requested criteria or question option is formulated into an entailment hypothesis. The model scores whether the input state entails, contradicts, or is neutral toward each option simultaneously.
• Strengths: Widely recognized as the most accurate encoder architecture on typed classification tasks, delivering 20.4ms median response times.

4. GLiClass (by Knowledgator)

Developed by Knowledgator, gliclass (439 MB) is an instruction-following zero-shot text classifier:

• Mechanism: GLiClass cross-encodes all target labels and criteria instructions concurrently in a single batch pass.
• Cost Invariance: In traditional classifiers, adding 20 possible intent categories dramatically increases compute. In GLiClass, the entire candidate space is evaluated in parallel, keeping latency fixed at ~14.7ms regardless of how many categories you supply.

Supported Question Types: Choice, Score, and Noul

Ollaya structures decision queries into three mathematical primitives that cover programmatic automation without prompt engineering:

• choice: An object mapping options to descriptions or a list of labels (2 to 255 options). Returns the selected choice, normalized confidence, and full probability distribution. About 125 options fit in laya:en‘s context, and 250 fit in laya:multilingual.
• score: An ordered list of 2 to 10 scale levels (e.g. ["Can wait", "Needs attention this week", "Needs attention today"]). Returns an expected scalar score level alongside confidence and class probabilities.
• noul: Evaluates the calibrated probability (0.00 to 1.00) that a propositional statement holds true, replacing clumsy boolean prompts with true Bayesian calibration.

Head-to-Head Benchmark: Ollaya vs. TypeSafe Jev

Independent benchmarks conducted across enterprise workloads paint a stark contrast between local open-weight runtimes and proprietary cloud APIs:

Why Calibration Trumps Raw Accuracy

In production systems, Expected Calibration Error (ECE) is vastly more critical than nominal accuracy. When a model returns 0.90 confidence across 1,000 cases, exactly 900 must be correct.

With TypeSafe Jev’s high ECE of 0.246, establishing automated thresholds is dangerous: an answer marked 88% certain might carry a 30% empirical error rate. With Laya running in Ollaya at an ECE of 0.081, engineers can confidently automate downstream operations based on numerical probability cutoffs.

Drop-In Replacement: Migrating from the TypeSafe SDK in 3 Seconds

Ollaya’s most calculated design decision is its wire-level compatibility. Ollaya implements the exact schema specifications of typesafe-sdk 0.7.1 across its /v1/systemone and /v1/models routes.

Developers who integrated TypeSafe Jev do not need to refactor their code, change their imports, or alter their schema criteria. Simply point the environment variables to your local Ollaya daemon:

# 1. Point the TypeSafe SDK to your local Ollaya daemon
export TYPESAFE_BASE_URL=http://localhost:11435

# 2. Supply a dummy local key (Ollaya runs locally with no license checks)
export TYPESAFE_API_KEY=local

# 3. Specify your default local open model
export TYPESAFE_DEFAULT_MODEL=laya

You can also query Ollaya’s native /api/decide endpoint directly via cURL, receiving typed answers, routing metadata, and durations in a single payload:

curl http://localhost:11435/api/decide -d '{
  "model": "laya",
  "state": "I was charged twice for my subscription this month. Please refund the second charge.",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": {
        "billing": "Payments, invoices and refunds",
        "technical": "Bugs, errors and outages",
        "account": "Login, profile and settings"
      }
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["Can wait", "Needs attention this week", "Needs attention today"]
    },
    "refund": {
      "type": "noul",
      "instructions": "The customer asks for money back."
    }
  }
}'

The response is returned in ~9ms flat with zero output tokens generated:

{
  "model": "laya:en",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.7781,
      "probabilities": { "billing": 0.8521, "technical": 0.0611, "account": 0.0868 }
    },
    "urgency": {
      "type": "score",
      "score": 1.1982,
      "confidence": 0.3418,
      "legend": { "0": "Can wait", "1": "Needs attention this week", "2": "Needs attention today" },
      "probabilities": { "0": 0.1203, "1": 0.5612, "2": 0.3185 }
    },
    "refund": {
      "type": "noul",
      "noul": 0.9127
    }
  },
  "usage": { "input_tokens": 118, "output_tokens": 0 },
  "routing": {
    "router": "laya:latest",
    "model": "laya:en",
    "route": "english",
    "reason": "English Latin text"
  }
}

Supercharging AI Agents: Native MCP and Sub-10ms Routing

While terminal commands and Python SDKs streamline developer operations, autonomous AI agents stand to benefit the most from Ollaya.

In multi-agent systems and developer assistants like Claude Code, Cursor Composer, or AutoGen swarms, agents waste significant time and API spend asking foundational routing questions:

• “Does this prompt require reading files or running a terminal command?”
• “Is this error a non-critical warning or a fatal syntax crash?”
• “Should this support inquiry be answered automatically or escalated to human review?”

Triggering an expensive autoregressive LLM for every minor decision wastes 3 to 10 seconds and hundreds of reasoning tokens per turn.

Native Model Context Protocol (MCP) Integration

Ollaya includes an out-of-the-box MCP server that communicates over stdio or local HTTP:

# Connect Ollaya to Claude Code
claude mcp add ollaya -- ollaya mcp

Or configure it in .cursor/mcp.json or claude_desktop_config.json:

{
  "mcpServers": {
    "ollaya": {
      "command": "ollaya",
      "args": ["mcp"]
    }
  }
}

By installing the official ollaya-decisions skill (npx skills add ollaya-dev/ollaya --skill ollaya-decisions), agents learn to make instantaneous, sub-10ms classification and routing calls before deciding whether to engage slow, expensive System 2 reasoning loops.

The Verdict: Why the Future of System 1 Is Open-Source

TypeSafe AI accurately identified the fundamental flaw of generative LLMs: software applications rarely need conversational prose; they need fast, deterministic, calibrated decisions.

However, TypeSafe chose the wrong delivery vehicle.

Decision models do not require massive GPU clusters. They range between 300 million and 2 billion parameters, meaning they execute with blistering speed on standard developer laptops, edge servers, or modest workstation GPUs. Charging metered cloud subscription fees for a model that executes in 9 milliseconds on consumer silicon was fundamentally unsustainable.

By combining an intuitive developer CLI, first-class open weights (Laya, Decider, GLiClass, NLI), wire-compatible TypeSafe SDK support, and native MCP agent tooling, Ollaya has established itself as the definitive open-source standard for System 1 decision models.

The days of paying monthly cloud bills for simple classification logic are over.