Ensemble (Mixture of Agents)

An ordinary request goes to one model. An ensemble request goes to several. Neural Router picks a handful of participant models, sends each of them your request at the same time, and asks each for its analysis and recommended approach rather than a finished answer. When they have all replied, a single pinned aggregator model receives your original messages plus every analysis, and writes the answer you actually receive.

Participants are chosen per request, and they are chosen by rank. The eligible pool is scored through the same routing path an ordinary request uses — measured outcome quality, PEQS grade, endpoint eligibility and cache-aware effective price all apply — and the ensemble draws from the top of that ranking. An ensemble that picked its members by any other rule would be routing twice, by two different sets of rules.

Over that ranking sits one preference: Neural Router takes the best-ranked endpoint from each distinct provider first, because three endpoints from one vendor cost the same as three perspectives while returning roughly one. Diversity is a preference, not a hard limit, so once every available provider is represented the ensemble backfills with the next best models — still in rank order — rather than running short.

The analyses are handed to the aggregator numbered, never named, so it judges the reasoning rather than deferring to a brand. Only the aggregator streams, and only the aggregator may call your tools; participants analyse and nothing more.

The default shape runs three participants plus the aggregator, so the honest planning figure is around four model calls per request. That is the trade. It is worth it when the answer is expensive to get wrong:

• Hard, open-ended engineering work— a migration plan, a tricky refactor, a design with real trade-offs, where disagreement between models is itself information.
• Answers a human will act on directly, where a subtly wrong answer costs far more than the extra tokens.
• Low-volume, high-stakes traffic— the review step at the end of a pipeline rather than every step in it.

It is plainly a bad fit for:

• Simple lookups and mechanical transforms. Classification, extraction, formatting, short factual answers — four models will agree with each other and you will have paid four times for the agreement.
• High-volume cheap traffic. If the point of the workload is unit cost, an ensemble is the wrong instrument; a routing objective like lowest-cost is the right one.
• Anything latency-sensitive. Every participant has to finish before the aggregator starts, so the response is gated on the slowest participant plus the aggregator’s own time. Interactive autocomplete, chat UIs with a typing indicator, and tight agent loops all feel worse on an ensemble.
• Anything you cannot measure. If you are not labelling outcomes, you have no way to know whether the extra spend bought you anything.

Do not take our word for the trade. The Fusion screen in the console compares ensemble against single-model answers on your own traffic, per task class, using your own outcome labels — and shows the cost multiplier you actually paid, not one we quoted.

It is a model id. Send it wherever you would send any other model id, on the same /v1/chat/completions endpoint:

curl https://api.neuralrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $NEURALROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nr/moa-max",
    "messages": [
      { "role": "user", "content": "Design a migration path off this schema without downtime." }
    ]
  }'

Because it is only a model id, every OpenAI-compatible client works unchanged — SDKs, coding tools, anything that lets you type a model name. There is no special request field and no separate endpoint.

Streaming works. Set stream: true as usual. The stream you receive is the aggregator’s; participants never stream, because their text is input to the answer rather than the answer.

{
  "model": "nr/moa-max",
  "stream": true,
  "messages": [ ... ]
}

Ensemble ids are listed in GET /v1/models alongside regular models when they are enabled for your deployment. If nr/moa-max is not in that list, requesting it returns an ordinary model-not-found error.

You are billed the sum of every model that ran. Each participant is charged at its own model's rate for its own tokens, and the aggregator is charged at its rate for its own tokens. There is no ensemble surcharge on top and no blended rate underneath — the invoice is a pass-through sum of the hops.

Because of that, the price shown for nr/moa-max in /v1/models is flagged indicative. A composition has no single true per-token rate: the figure is the aggregator's own rate plus one median catalog rate per participant, published as an estimate rather than a quote your invoice would fail to match. Which participants ran, and therefore what the request cost, is decided per request.

Every hop is recorded as its own usage record, so Logs and Usage in the console list each participant and the aggregator separately, and a monthly total still reconciles back to individual calls. See Pricing & service types.

Two smaller cost notes. A participant that replies but produces nothing usable — for example one that tries to call a tool instead of analysing — is discarded from the synthesis, but it did the work and is still billed. And the semantic cache is not consulted for ensemble requests, so X-NR-Cache is always none.

Nothing special is required: set the model to nr/moa-max and the tool gets an ensemble. In Claude Code that is ANTHROPIC_MODEL="nr/moa-max"; in Cursor, add nr/moa-max as a custom model; in Cline, Continue, Aider, and OpenCode it is the model id field you already fill in. Full setup for each tool is on Coding tools.

Tool calling still works. The aggregator is the model that sees your tools and issues the calls, so from the tool's side the exchange looks exactly like talking to one strong model. Given the cost and the added latency, most teams point one profile or one command at nr/moa-max for the hard steps rather than routing a whole agent loop through it.

Response headers describe the hop that answered you, not the whole ensemble:

X-NR-Model: claude-sonnet          # the aggregator that wrote this answer
X-NR-Route-Reason: moa: nr/moa-max
X-NR-Trace-Id: nrq_8f2a1c4b
X-NR-Cache: none

• X-NR-Model is the aggregator— the model that wrote the answer in your hands.
• X-NR-Route-Reason names the ensemble that served the request — and says so differently when the ensemble did not actually run, which is how a budget fallback reaches your client (see When your budget is low below).
• X-NR-Trace-Id is the id to quote in support, and the id to attach outcome labels to. A label on an ensemble request is recorded against the aggregator's answer, which is what makes the Fusion comparison meaningful.

The participants are on the routing trace, not in the headers: the trace records one hop per participant plus the aggregator, so the receipt names every model the request paid for rather than only the one that spoke. The per-hop usage records carry the same trace id, tagged with the part each played.

The aggregator is pinned in configuration rather than selected per request, so it cannot be swapped for whichever model happens to satisfy your policy. Both halves of the ensemble are held to the same rules your normal traffic is held to — zero data retention, compliance classes, and your API key's allowed-models list — and the check happens before anything is dispatched, so a refusal never arrives after you have paid for participants.

If the configured aggregator cannot satisfy those rules, the request is refused outright with a 403. It is not quietly downgraded to a different aggregator and not run anyway:

HTTP/1.1 403 Forbidden

{
  "error": {
    "message": "ensemble nr/moa-max is unavailable under this workspace's policy: this workspace requires zero data retention, which cannot be established for the model that completes this ensemble"
  }
}

The refusal reasons are:

• Key restriction. The API key you used is not permitted to use the model that completes the ensemble.
• Workspace routing policy. Your policy names a candidate list that does not include the aggregator. An empty list still means “the whole catalog”, exactly as it does on the normal path, so this only bites a workspace that has deliberately narrowed its models.
• Aggregator not eligible to serve. The pinned model is quarantined — the lever an operator or the breach classifier pulls to take an endpoint out of service. An unknown model id lands here too. A transient degraded health dip does not: the tolerant service types serve through it.
• Jurisdiction. Your policy blocks the jurisdiction the aggregator sits in, or that jurisdiction cannot be attested under an active blocklist.
• Zero data retention. Your workspace requires it and retention cannot be established for the aggregator. See Zero-retention & residency.
• Compliance class. Your workspace requires a class for this request's content that the aggregator does not carry.

The participant half degrades instead of refusing, which is the right behaviour for the same policies applied to models that are chosen rather than pinned. Under a narrow policy the ensemble may simply shrink to fewer participants; if no compliant participant is left, the aggregator answers on its own. The same is true of failure: if every participant errors, the aggregator still answers and the request still succeeds. An ensemble that failed closed would be less reliable than the single model it is meant to beat.

A workspace budget has two thresholds, and an ensemble — the request shape most worth slowing, because it bills several hops — answers to both.

At the hard cap the request is refused outright, before the ensemble is looked at, with the same 402 any other request gets. No participant is selected, nothing is dispatched, and no usage event is written:

HTTP/1.1 402 Payment Required

{
  "error": {
    "message": "workspace budget hard cap reached — raise the budget or wait for the window to reset"
  }
}

In the warning zone the ensemble falls back to the aggregator alone. An ordinary request in that state has its objective dropped to lowest-cost; an ensemble instead becomes a normal single-model request served by the pinned aggregator. No participants are selected, none are dispatched, and exactly one hop is billed — so you pay 1x rather than roughly 4x.

It still succeeds, and that is the point. Refusing would be a harsh way to discover your budget is running low, and it would break a running agent loop mid-task over a soft threshold. Merely making the participants cheaper would have kept the multiplier that is the cost problem in the first place: cheaper participants still cost several times a single call. Dropping to one call is the only change that meaningfully slows the spend.

The fallback is never silent. The answer came from one model instead of four, so the reason says so — on the routing trace, and in the X-NR-Route-Reason response header your client can read without opening the console:

X-NR-Model: claude-sonnet
X-NR-Route-Reason: moa: nr/moa-max downgraded to the aggregator alone: workspace budget degraded
X-NR-Trace-Id: nrq_8f2a1c4b
X-NR-Cache: none

The trace also carries the budget state that caused it, the same field an ordinary degraded request is stamped with.

Order matters here: the policy check above runs first. A low budget is not a reason to relax a gate — a fallback request is served by the aggregator alone, so an aggregator your retention guarantee, your routing policy or your key forbids is now the entire request rather than one hop of it. It is refused, not downgraded.

If your workspace has an enterprise reservation, its ensembles honour it exactly as ordinary requests do: the reservation is resolved per request and the participant ranking runs under your routing class, not under a generic standard one.

An ensemble occupies several endpoints at once, which makes it more likely than a single request to land on reserved capacity — so this matters in both directions. An ensemble cannot squat on capacity another workspace paid for, and your own ensemble gets the capacity you bought. Where an isolated tier confines routing to your reserved endpoints, the fan-out is confined with it — picking several endpoints at once is no longer a way out of isolation.

If reservation metadata cannot be read, the request fails rather than quietly falling back to the standard class. Steering an enterprise workspace on metadata we could not read is how reserved capacity gets resold by accident.

Two notes. A workspace without a reservation is on the standard class and the shared pool, which is the behaviour it always had. And a budget-fallback request skips participant ranking altogether — there are no participants to choose — so it is dispatched as the pinned single call it has become. See Service tiers & savings.

Browse ids on Models, compare it against ordinary routing on Model routing, and check the measured delta on your own traffic under Fusion in the console.