speculative fan-out

The four published patterns

Speculative fan-out: send many questions in a single call, including speculative ones, and let your code decide what is relevant. Benefits listed by TypeSafe are cost and speed. The smart-home demo is the worked example. For "Turn off all of the lights in the house" you still ask category, domain, device, and action together, including the action question before you know the utterance is about lights. Sequential calls would wait. Parallel evaluation plus a filter in code is the point.

Confidence-gated routing: use confidence as a second axis for whether to act. Benefits are reliability and safety. A label without a confidence gate will fire on a flat distribution. Pair the Choice or Score with thresholds that scale with risk, and keep Noul on its own axis.

Composite scoring: break a judgment into atomic scores and combine weights in code. Benefits are cost, reliability, and speed. Do not ask Jev for one giant 1-100 quality number. Score evidence, policy fit, and severity separately, then add. Intent routing: classify intent and route to logic, an LLM, or a human. Benefits are cost and speed. Jev decides which handler runs; it does not become the handler that writes the reply.

Example decisions from the use-case map

TypeSafe's use-case map lists industry examples rather than a second primitive set. Customer support classifies tickets, detects urgency and refunds, and routes queues. Insurance claims classify FNOL notes and escalate high-risk files. Financial crime scores alert narratives and routes ambiguous KYC. Legal and compliance detect missing clauses and prohibited claims. E-commerce marketplaces normalize listings and flag counterfeit signals.

Moderation applies company-specific criteria with severity. Advertising checks brand safety and ad-to-landing alignment. Gaming moderates chat, scores frustration, and routes player support. Recruiting scores resumes against explicit job-related criteria. Lead generation matches profiles to an ICP. Model routing chooses which LLM receives a prompt. LLM guardrails detect jailbreaks, injection, and policy violations.

Search and retrieval score query-to-candidate relevance. Feature extraction turns language into probabilistic features for classical ML. Risk assessment converts incident notes into indicators. Graphs classify relations and detect contradictions. Semantic code linting runs team-specific checks in CI. Demand forecasting extracts intent and urgency features. Each row is a decision shape, not a hosted workflow. You still own the code that acts.

Compose in code, not in one prompt

Fan-out plus confidence routing is the usual production pair: ask more than you need, drop the irrelevant Nouls, then only auto-act when Choice confidence clears a high bar. Composite scoring sits under that when a single Score would hide three different judgments. Intent routing is the front door that decides whether Jev, an LLM, or a human sees the rest of the request.

Eval families published on evals.typesafe.ai include Security Incidents, Agent Trace Observability, Invoice Processing, and Customer Service. TypeSafe's homepage 193.6x / 444.6x figures come from those workflow evals versus Astra and Fable averages. Use them as TypeSafe's published comparison, not as a number this wiki re-ran.

If a pattern you need is not in the four, the primitives still compose. Keep new control flow in your repository. File a note with TypeSafe if you want it on the docs index. Extra compositions stay in application code until TypeSafe publishes another named pattern.

Picking a pattern for a workflow

Start with speculative fan-out whenever several questions share one state and some of them might not apply. That is the default for assistants, ticket bots, and anything that looks like a switchboard. Add confidence-gated routing the moment an answer can fire a side effect: refund, ban, wire, deploy.

Use composite scoring when a single Score would mix unlike dimensions. Evidence quality, policy fit, and customer severity are three Scores, then a weighted sum you own. Use intent routing at the edge of a mixed system that also has an LLM or a human queue, so Jev decides the destination instead of writing the destination's work.

The 18 example decisions on this page are TypeSafe's map, not extra endpoints. Customer support, claims, guardrails, search, and gaming all still call POST /v1/systemone with Choice, Score, and Noul. Copy the decision shape, then write the criteria from your policy text rather than from a generic prompt library.

Fan-out is the default. Add confidence gates when a label can spend money or ban a user. Split Scores when a single legend would mix unlike axes. Put intent routing at the boundary with an LLM. Those four sentences are the whole published pattern set.

The use-case map is eighteen decision shapes, not eighteen APIs. Each row still becomes one or more questions on POST /v1/systemone. Copy the shape into your domain, then write criteria from your policy. A generic moderation prompt that is not your policy will not match this map.

Speculative fan-out is the pattern whose name is also this page's H1, because it is the one TypeSafe leads with. The other three sit under it as ways to use the answers after the single call returns.

Eighteen example decisions share one family name, automation, because they are software judgments, not a second product line.