Unsloth Decision API: Run Laya via Local Jev Protocol

When Unsloth AI announced local execution support for Convai Innovations’ open-weight Laya decision models, the immediate reaction focused on lightweight inference. But the actual engineering coup is wire protocol interoperability: Unsloth has integrated a drop-in local emulator of TypeSafe’s Jev API (Unsloth Decision Laya Documentation). Operating on a 421M-parameter ModernBERT backbone, the local daemon processes structured decision criteria in 33 milliseconds on consumer silicon—completely offline, with zero token billing, and without transmitting confidential enterprise data over cloud sockets.

For software teams orchestrating multi-agent systems, this release directly addresses an industry-wide anti-pattern. Over the past two years, developers have routinely routed deterministic triage, intent tagging, and tool branching through massive autoregressive foundation models. As we documented in our teardown of Ollaya and the non-autoregressive decision landscape, using a 70B generative LLM to determine whether an incoming support ticket requests a refund is the computational equivalent of using an industrial blast furnace to toast a single slice of bread. It wastes compute, inflates latency budgets, introduces nondeterministic JSON parsing failures, and leaks sensitive customer context to external model vendors.

Autoregressive Slop vs. Single-Pass Forward Projections

The foundational difference between a generative LLM and a decision engine lies in the inference decoding loop. When prompting Claude or GPT-4o for structured classification, the model must decode tokens autoregressively—one token at a time—through dozens of transformer layers. A simple decision takes between 400ms and 1,800ms of compute time, and the resulting payload must be safely extracted from markdown code fences or parsed via fragile regex handlers.

In contrast, Convai Innovations’ Laya architecture (Hugging Face model weights) abandons the generative decoding loop entirely. Built upon the ModernBERT architecture (arXiv:2412.13663), Laya performs bidirectional cross-attention across the input state and all candidate criteria simultaneously. It directly projects classification logits onto pre-trained decision heads in a single forward pass.

The critical telemetry metric returned in every Unsloth daemon payload is "output_tokens": 0. There is no string formatting, no temperature drift, and zero probability of hallucinated JSON formatting. You receive calibrated float probabilities directly mapped to discrete decisions.

The Wire Protocol: Drop-In /v1/systemone Compatibility

Unsloth’s implementation binds a lightweight local daemon to port 8888, fully exposing the wire protocol defined by TypeSafe’s Jev API. Migrating an existing cloud production harness does not require modifying SDK calls or refactoring validation schemas—developers simply redirect the client environment variable to the local endpoint:

# Redirect official TypeSafe client to local Unsloth daemon
export TYPESAFE_BASE_URL="http://localhost:8888"
export TYPESAFE_API_KEY="sk-unsloth-local-key"

from typesafe import TypeSafeClient

client = TypeSafeClient()

# Non-autoregressive single-pass decision
response = client.decide(
    model="laya",
    state="Client charged twice on subscription renewal. Requesting reversal.",
    questions={
        "department": {
            "type": "choice",
            "criteria": {"billing": "refunds, charges, invoices", "support": "bug reports"}
        },
        "escalate": {
            "type": "noul",
            "instructions": "Does this require priority supervisor intervention?"
        }
    }
)

# Calibrated output in ~33ms:
# response.answers["department"].choice -> "billing" (prob: 0.9841)
# response.answers["escalate"].noul     -> 0.8920

Notice how cleanly the request is structured. There is no chat history, no conversational fluff, and no system prompt boilerplate. You pass the application state, declare typed questions, and receive bounded evaluation objects.

Core Evaluation Primitives: Noul, Choice, and Score

The TypeSafe wire specification relies on three formal mathematical primitives that cover virtually all agentic routing topologies:

• noul (Bayesian Proposition Truth): Evaluates the probability from 0.0 to 1.0 that an arbitrary natural-language assertion holds true. Downstream controllers can execute deterministic threshold gates (e.g. if answer.noul > 0.85) without wrestling with LLM string matching.
• choice (Categorical Softmax Selection): Compares the state context against up to 255 discrete candidate keys. It returns the winning category while preserving the complete probability distribution across all valid labels, allowing applications to detect ambiguous edge cases when top contenders are evenly split.
• score (Ordinal Likert Regression): Maps input text onto an ordered integer scale of 2 to 10 discrete intervals. It outputs an expected continuous value alongside per-interval probabilities, enabling smooth thresholding for sentiment analysis, risk scoring, or priority tiering.

Hardware Specifications & Memory Footprint

Unlike generative LLMs that require 8GB to 24GB of VRAM just to serve quantization runtimes, Laya’s compact parameter footprint allows it to run comfortably in constrained environments, including local homelabs and edge devices:

Production Friction Points & Calibration Hazards

While wire compatibility makes migration trivial at the syntax level, our systems audit identified three critical behavioral differences that production teams must account for:

Architectural Takeaway: Decoupling Reflexes from Reasoning

The broader significance of Unsloth’s release is the emerging architectural pattern for next-generation agent swarms: the separation of System 1 reflexive gating from System 2 deliberative reasoning. In production harnesses like local autonomous CLI agents and sandboxed execution environments, having a 421M-parameter local decision engine act as an always-on spinal cord cuts latency and operational costs dramatically.

Instead of burning expensive tokens on frontier reasoning models like Claude Sonnet 5.5 or OpenAI GPT-4o for routine classification, teams can now run high-velocity Laya routing on-device. When edge criteria dictate that a complex multi-step task is actually required, the system cleanly delegates upstream to frontier models. By bringing TypeSafe wire parity to local hardware, Unsloth has delivered a practical, zero-marginal-cost cornerstone for private, resilient AI infrastructure.

Frequently Asked Questions