multi-router.aidocs

Routing

Adaptive Router

One router over a model group, with rules on the feature, customer and requested model that choose the model for each request.

An Adaptive Router is a router that serves many features and customers from one model group. Its rules match values the request carries, such as the feature that made the call, the customer and the requested model, and each rule sends the request to one group member, to a fallback list of members, or to a weighted split between members. A request that names no model is treated as asking for the router's default model, and a request that matches no rule is served the model it asked for. The router calls each provider with the organization's provider key for that provider, and records the rule that chose the model in Traffic.

Endpoints

An Adaptive Router serves the two inference endpoints in their own formats. A request selects the router with the X-MR-Router header, or by naming the router in the model field, such as multi-router/rt_9Hk3Ws7PzQ:

EndpointFormat
POST /v1/chat/completionsOpenAI Chat Completions
POST /v1/messagesAnthropic Messages

The request is translated to the format of the provider that serves it, as for a static router. Coding tools that send OpenAI Chat Completions connect through an Adaptive Router; see Agents & Editors.

Model group and members

An Adaptive Router serves the members of one model group, set in routing.model_group. Each member names a model, a provider, a variant and a region, so the same model can appear several times, for example Claude Opus 5 on Anthropic and on AWS Bedrock. A member used by a rule or the default must have a variant and a region. See Model Group.

Rules and the default name members by their model slug, with the provider, variant and region added only when the slug alone matches more than one member:

Member referencesjson
{ "model": "anthropic/claude-sonnet-4.5" }
{ "model": "anthropic/claude-opus-5", "provider": "bedrock" }
{ "model": "anthropic/claude-opus-5", "provider": "bedrock", "variant": "geo", "region": "eu-central-1" }

Each member can be switched on or off. A switched-off member is listed in routing.adaptive.disabled; it is skipped in fallback lists and weighted splits, and a request for a model that only switched-off members serve is refused with model_disabled. In the dashboard, the switch next to each member saves at once.

Credentials

An Adaptive Router uses organization provider keys. routing.adaptive.credentials holds one key for each provider in the group, and member_keys can give a single member its own key. Every switched-on member's provider needs a key, and a router uses at most 10 distinct keys. Each key must be an enabled key of that provider in the router's organization.

Credentialsjson
"credentials": {
  "anthropic": { "source": "organization", "provider_key_id": "llpk_4Vd8Qm2RtL" },
  "openai": { "source": "organization", "provider_key_id": "llpk_7Ns2Xc5PwK" }
}

Developer credentials, which pass each person's own provider login through, are not available on an Adaptive Router: "source": "developer" returns credential_not_allowed.

Rules

Rules are checked in order, and the first rule whose conditions all match plans the request. A router holds up to 200 rules. Each rule has a name of 1 to 64 characters, unique in the router, at least one condition, a choice in then, and optionally a prompt binding in prompt. The conditions are:

ConditionMatchesValue in the request
featureThe feature label, exactly.The X-MR-Feature-Id header, up to 200 characters, such as support/triage.
feature_prefixThe feature label, by its start.The X-MR-Feature-Id header. support/ matches support/triage and support/reply.
customerThe customer, exactly.The user field on Chat Completions, or metadata.user_id on Messages.
modelThe requested model, with optional provider, variant and region.The model field.

Conditions match values the caller sends, so rules steer requests and do not restrict access: any caller of the router can send any feature, customer or model. Use API keys and router access to control who can call the router; see Router Access.

Choices

The then of a rule holds exactly one of three choices. The default holds single only; see The default and the requested model.

ChoiceMembersBehavior
single1The member serves the request.
fallback2 to 10The members are tried in order, and the request moves to the next member under the same conditions as a fallback router.
weighted2 to 10One member is chosen by weight, a whole number from 1 to 10,000. split_by sets what keeps a caller on one member.
split_byBehavior
session (default)Requests with the same X-MR-Session-Id header stay on one member. A request without the header is chosen on its own.
customerRequests for the same customer stay on one member.
requestEvery request is chosen on its own.

A member appears at most once in a choice.

The default and the requested model

Each request is planned in this order:

  1. The model field names the requested model. When it names no model (the router's id or alias, or an empty value) and the router has a default, the default's model, with its provider, variant and region, becomes the requested model.
  2. Rules are checked in order, and the first rule that matches plans the request. A model condition matches the requested model from step 1, so a rule on the default's model also matches a request that names no model.
  3. When no rule matches, the requested model serves. When it came from the default, the default's member serves, with the default's prompt binding if it names one. When the request named it, the first switched-on member that serves that model serves the request.
  4. A request that names no model, on a router with no default, that matches no rule is refused with router_misconfigured.

The default is one model, in routing.adaptive.default. Its then holds single; setting or changing the default to a fallback or weighted split returns 400 default_must_be_single. A router saved with a fallback or weighted default before this rule keeps it: it is returned as saved, keeps serving, and updates that leave it unchanged are accepted. For a fallback or a split on requests that name no model, add a rule whose model condition is the default's model:

A default, and a fallback for requests that name no modeljson
"rules": [
  {
    "name": "default-fallback",
    "when": { "model": { "model": "anthropic/claude-sonnet-4.5", "provider": "anthropic" } },
    "then": { "fallback": [
      { "model": "anthropic/claude-sonnet-4.5", "provider": "anthropic" },
      { "model": "anthropic/claude-sonnet-4.5", "provider": "bedrock" }
    ] }
  }
],
"default": {
  "then": { "single": { "model": "anthropic/claude-sonnet-4.5", "provider": "anthropic" } },
  "prompt": "support-reply"
}

The model field accepts the router's id or alias, a catalog slug such as anthropic/claude-sonnet-4.5, or a provider's own model id such as claude-sonnet-4-5. A model the group does not serve returns 400 model_not_in_router, with the list of models the router serves. Traffic records the model field as the request sent it. The dashboard always keeps a default, set first to the group's first model.

How the dashboard shows rules

The router's page shows the rules, and its editor writes them, as a short config. A model is written in full as <publisher>/<model>@<provider>:<variant>|<region>. A model condition shows only the parts it names, and matches on those parts. Conditions use == and starts_with, joined by AND. fallback[...] lists members tried in order; weighted[...] lists each member with its weight, followed by its split. Long lists wrap one member per line.

Rules as the dashboard shows themtext
if: feature == chat AND customer == acme
then:
  model = anthropic/claude-haiku-4.5@anthropic:standard|global
  prompt = reply

if: feature starts_with email/
then:
  model = fallback[anthropic/claude-sonnet-4.6@bedrock:geo|eu-central-1, anthropic/claude-sonnet-4.6@bedrock:geo|eu-west-1]

if: model == anthropic/claude-opus-4.5@anthropic:standard|global
then:
  model = weighted[95: anthropic/claude-sonnet-4.6@anthropic:standard|global, 5: openai/gpt-5.5@openai:standard|global]
  split = session

default:
  model = anthropic/claude-sonnet-4.6@anthropic:standard|global
The same rules in routing.adaptivejson
"rules": [
  {
    "name": "acme-chat",
    "when": { "feature": "chat", "customer": "acme" },
    "then": { "single": { "model": "anthropic/claude-haiku-4.5", "provider": "anthropic" } },
    "prompt": "reply"
  },
  {
    "name": "email",
    "when": { "feature_prefix": "email/" },
    "then": { "fallback": [
      { "model": "anthropic/claude-sonnet-4.6", "provider": "bedrock", "region": "eu-central-1" },
      { "model": "anthropic/claude-sonnet-4.6", "provider": "bedrock", "region": "eu-west-1" }
    ] }
  },
  {
    "name": "opus-trial",
    "when": { "model": { "model": "anthropic/claude-opus-4.5", "provider": "anthropic", "variant": "standard", "region": "global" } },
    "then": { "weighted": {
      "members": [
        { "member": { "model": "anthropic/claude-sonnet-4.6", "provider": "anthropic" }, "weight": 95 },
        { "member": { "model": "openai/gpt-5.5" }, "weight": 5 }
      ],
      "split_by": "session"
    } }
  }
],
"default": { "then": { "single": { "model": "anthropic/claude-sonnet-4.6", "provider": "anthropic" } } }

A router with no default shows default: with a note that a request that names no model and matches no rule is refused.

Prompt bindings

An Adaptive Router holds up to 20 named prompt bindings in routing.adaptive.prompts. Each name is 1 to 64 characters of a-z, 0-9, - and _, and each binding has the same shape as a static router's prompt binding. A rule, or the default when it serves, applies a binding by naming it in prompt. A request can choose a binding itself with the X-MR-Prompt header, which names a binding, such as support-reply, or a binding and version, such as support-reply@3, and takes precedence over the rule. A name that is not bound on the router returns 400 prompt_not_bound, and a version the binding cannot serve returns 400 prompt_version_not_available. See Prompt Binding.

Headers and Traffic

Every response planned by an Adaptive Router carries the X-MR-Rule header, with one of these values:

ValueMeaning
The rule idThat rule planned the request.
requested_modelNo rule matched, and the model the request named served it.
defaultNo rule matched, and the request named no model, so the default's model served it.

Traffic records the rule id and name, the model group and the feature label with each request, alongside the provider, model, variant, region and cost of the member that served it. The Traffic API filters by rule_id, feature_id and feature_prefix. See Traffic.

Creating and editing an Adaptive Router

To create an Adaptive Router in the dashboard:

  1. On the Routers page, select New router, and under Routers choose Adaptive Router.
  2. Enter a Name, and choose a model group. The group's models are listed with a switch each, and the first one is the default; Set as default moves it.
  3. Under Credentials, choose an Organization key for each provider in the group. A missing key links to the Provider Keys page.
  4. Optionally add prompt bindings and routing rules, then select Create.

On the router's page, Edit switches the Rules and Routing panels into edit mode together. Rule values are edited in place, rules can be added, reordered and deleted, and the model group, keys and prompt bindings are saved with the same Save. Discard restores the saved configuration. When another change was saved in the meantime, a banner offers Reload.

With the API, send POST /v1/routers with kind adaptive_router and routing: model_group and adaptive. A new rule has no id; Multi-Router assigns one:

Create an Adaptive Routerbash
curl -X POST https://api.multi-router.ai/v1/routers \
  -H "Authorization: Bearer $MR_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "adaptive_router",
    "name": "Product AI",
    "routing": {
      "model_group": "mg_4kT9wQx2Lp",
      "adaptive": {
        "credentials": {
          "anthropic": { "source": "organization", "provider_key_id": "llpk_4Vd8Qm2RtL" },
          "openai": { "source": "organization", "provider_key_id": "llpk_7Ns2Xc5PwK" }
        },
        "rules": [
          {
            "name": "support",
            "when": { "feature_prefix": "support/" },
            "then": { "fallback": [
              { "model": "anthropic/claude-haiku-4.5" },
              { "model": "openai/gpt-5-mini" }
            ] }
          },
          {
            "name": "summaries-trial",
            "when": { "feature": "docs/summarize" },
            "then": { "weighted": {
              "members": [
                { "member": { "model": "anthropic/claude-sonnet-4.5" }, "weight": 90 },
                { "member": { "model": "openai/gpt-5" }, "weight": 10 }
              ],
              "split_by": "session"
            } }
          }
        ],
        "default": { "then": { "single": { "model": "anthropic/claude-sonnet-4.5" } } }
      }
    }
  }'

A create always makes an enabled router with no tags; tags are added afterwards. PATCH /v1/routers/{id} changes name, description, enabled, rate_limits, observability, routing.model_group and routing.adaptive. routing.adaptive is replaced as a whole document: a rule sent back with its id keeps it, a rule without an id is new, and a rule left out is deleted. Send the router's ETag in If-Match; a stale version returns 412 with the current router.

A request through the routerbash
curl https://api.multi-router.ai/v1/chat/completions \
  -H "Authorization: Bearer $MR_API_KEY" \
  -H "X-MR-Router: rt_9Hk3Ws7PzQ" \
  -H "X-MR-Feature-Id: support/triage" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rt_9Hk3Ws7PzQ",
    "user": "acme",
    "messages": [{ "role": "user", "content": "Where is my order?" }]
  }'

What it does not do yet

Not availableUse instead
OpenAI Responses, POST /v1/responsesA Codex Proxy for OpenAI models.
Developer credentials, such as a ChatGPT or Claude subscriptionA Claude Proxy with Developer Credentials, or a Codex Proxy with each developer's own OpenAI API key.
The model list, GET /v1/models with the router headerName the model ids in the tool's settings. The request returns 404.

Errors

Creating or changing an Adaptive Router returns 400 for an invalid configuration, with the code, the field and a location that points at the value, such as routing.adaptive.rules[2].then.fallback[1]:

ErrorCause
member_not_in_groupA member reference matches no member of the model group.
ambiguous_memberA member reference matches several members. Add provider, variant or region.
member_needs_tupleA member used by a rule or the default has no variant or region in the group.
default_must_be_singleThe request sets or changes the default to a fallback or weighted split. The default is one model; use a rule instead.
fallback_too_short, weighted_too_short, too_many_choice_membersA fallback or weighted split has fewer than 2 or more than 10 members.
invalid_weightA weight is not a whole number from 1 to 10,000.
duplicate_choice_memberA member appears twice in one choice.
rule_without_condition, invalid_conditionA rule has no condition, or a condition value is empty, too long or not printable.
invalid_rule_name, duplicate_rule_name, too_many_rulesA rule name is missing, over 64 characters or repeated, or there are more than 200 rules.
prompt_not_bound, invalid_binding_name, too_many_prompt_bindingsA rule names a binding the router does not hold, a binding name is invalid, or there are more than 20 bindings.
provider_credential_missing, provider_key_requiredA switched-on member's provider has no key, or an organization credential names no key.
provider_key_not_found, provider_key_disabled, provider_key_mismatchA key is not in the organization, is disabled, or belongs to another provider.
credential_not_allowedA credential uses the developer source.
model_group_emptyThe model group has no members.

Requests through an Adaptive Router return the following errors:

ErrorStatusCause
model_not_in_router400The model field names a model the group does not serve.
model_disabled400Only switched-off members serve the requested model.
router_misconfigured400No rule matched, the request named no model and there is no default, or every member of the matched choice, or the default's model, is switched off.
prompt_not_bound, prompt_version_not_available400The X-MR-Prompt header names a binding the router does not hold, or a version the binding cannot serve.
router_disabled400The router is disabled.
Static RouterFallback Router
Was this helpful?