multi-router.aidocs

Providers & Models

Overview

How Multi-Router organizes models, providers, variants and regions, and how it uses your provider keys to route each request.

Multi-Router sends each request to a model provider, such as Anthropic, OpenAI, AWS Bedrock or Google Vertex, using credentials from your own account with that provider. A router decides which model serves a request and on which provider, variant and region; the model catalog supplies the model id and price for that choice; and a provider key supplies the credentials.

Setting up routing takes three steps. You add a provider key for each provider you use (see Provider Key), create a router that targets a model on one of those providers (see Routing), and send requests to the router with a Multi-Router API key (see API Access).

Routers and API keys

Multi-Router separates what a request does from who sends it. A router is a named object in an organization that holds the behavior for the requests sent to it: which model and provider serve them, which provider key is used, which variant and region they run in, and the prompt, parameter defaults and logging applied to them. An API key holds identity: it identifies the organization a request belongs to and what the caller is permitted to do. Because an API key carries no routing configuration, the behavior of every request is set by the router it names, and changing a router changes the behavior for every caller of that router without new API keys or changes to application code.

Every router has a type, and each type encapsulates one routing behavior. A request names a router in its model field, and the router's type determines what happens next:

Router typeBehavior
Static routerSends every request to one model on one provider, in a set variant and region, with a set provider key.
Fallback routerTries its member routers in order and returns the first successful response.
Weighted routerSplits traffic across its member routers by weight.
Claude ProxyServes Claude Code and Claude Desktop, forwarding their requests to Anthropic, AWS Bedrock or Google Vertex in the native format.
Adaptive RouterServes many features and customers from one model group, with rules that choose the model, a fallback list or a weighted split for each request.
Codex ProxyServes Codex CLI, forwarding OpenAI Responses requests to OpenAI with the organization's OpenAI key or the developer's own.
Gemini ProxyServes Gemini CLI, forwarding Gemini requests to Google with the organization's Gemini key.

An alias is a stable name that points to a router. A request can name the alias in place of the router, and repointing the alias moves every caller to a different router at once. See Routing.

A static router holds the following behavior, all of which can change without changing its callers:

SettingEffect
ModelThe model that serves requests.
ProviderThe provider that runs the model.
VariantThe way the provider offers the model, such as a Bedrock inference scope or a dated snapshot.
RegionWhere inference runs, which with the variant sets the model id and price.
Provider keyThe credentials and provider account used, or the organization's default key.
Model sourceWhether a request selected with the x-mr-router header uses the request's model or the router's model.
Prompt bindingA saved prompt applied to every request, merged with or replacing the caller's system message.
Parameter defaultsValues such as temperature, maximum tokens, top_p and stop sequences applied when the request omits them.
Request loggingHow much of each request Traffic records, from none to full payloads.
TagsLabels that group routers, used by budgets and saved views.
Enabled stateA disabled router refuses every request, which stops its traffic at once.

An API key holds the following identity and permissions:

PropertyEffect
OrganizationThe organization whose routers, provider keys and budgets the request uses.
Inference accessAny router, Specific routers, or None: the routers the key can send requests to.
Management scopesRead, Write or Delete access to routers, prompts, model groups, budgets and other resources through the API.

The same router can serve several applications, each with its own API key, and one application can use several routers by naming a different router in each request. See Routing and API Access.

The building blocks

The following terms are used throughout the Providers & Models documentation.

TermDefinition
ModelA model identified by its slug, such as anthropic/claude-haiku-4.5, independent of where it is served.
PublisherThe company that built a model, which forms the first part of its slug.
ProviderA company or cloud platform that serves models, such as Anthropic, OpenAI, AWS Bedrock or Google Vertex. Providers are also called vendors.
OfferingOne provider and variant of a model, with the regions where it is available.
VariantOne way a provider offers a model, such as its standard endpoint or a Bedrock global inference profile.
RegionA location where a variant runs, with its own model id and price.
Model idThe id Multi-Router sends to the provider for a catalog entry, shown as Wire id in Traffic.
Model catalogThe list of models Multi-Router can serve, organized by model, provider, variant and region, with the model id, list price, capabilities and lifecycle status of each entry.
Provider keyThe credentials for one of your provider accounts, which Multi-Router uses to call that provider on your behalf.
Default keyThe provider key an organization uses for a provider when a router names none.
Model groupA named list of approved models, each with a provider and optionally a variant and region, used by Adaptive Routers, Claude Proxies and tools policies.
Static routerA router that sends requests to one model on one provider.
AliasA stable name that points to a router and can be repointed without changing callers. An alias is not a router type.
Fallback routerA router that tries its member routers in order until one succeeds.
Weighted routerA router that splits traffic across its member routers by weight.
Claude ProxyA router that forwards Claude Code and Claude Desktop traffic to Anthropic, AWS Bedrock or Google Vertex in its native format.
Adaptive RouterA router over a model group whose rules choose the model, a fallback list or a weighted split for each request.
Codex ProxyA router that forwards Codex CLI traffic in the OpenAI Responses format to OpenAI.
Gemini ProxyA router that forwards Gemini CLI traffic in the Gemini format to Google.
TrafficThe dashboard log of requests, with the router, provider, model, variant, region, tokens, cost, latency and status of each.
BudgetA spending limit that refuses requests once it is reached. See Budgets & Spend Control.

How models are organized

The model catalog organizes each model in four levels: the model itself, the providers that serve it, the variants each provider offers, and the regions where each variant runs. Together, these four values identify exactly one entry in the catalog. The catalog entries for anthropic/claude-haiku-4.5 are organized as follows:

Model
Provider
Variant
Regions
anthropic/claude-haiku-4.5
anthropic
standard
global
20251001
global
bedrock
geo
us-east-1, eu-central-1, ap-northeast-1 and more
global
us-east-1, eu-central-1, ap-northeast-1 and more
vertex
standard
global, us, eu, us-east5

Models and slugs

A slug is the unique name of a model in Multi-Router, in the form publisher/name, such as anthropic/claude-haiku-4.5 or openai/gpt-5. The publisher and the provider are separate: Anthropic publishes anthropic/claude-haiku-4.5, and Anthropic, AWS Bedrock and Google Vertex all serve it. A slug identifies the model regardless of which provider serves it, is matched without regard to case, and is the name you use for a model when configuring routers and model groups.

GET /v1/models returns the chat models your organization can call: models that are active or deprecated and have an offering that is not retired from a provider for which the organization holds an enabled provider key. See Model Catalog.

List available modelsbash
curl https://api.multi-router.ai/v1/models \
  -H "Authorization: Bearer $MR_API_KEY"

Providers

A provider is the platform that runs the model and bills for its use. When a model is available from several providers, the provider determines which account is billed, the price, the regions available and the date the model is retired, and in some cases a smaller context window than the model supports elsewhere. Multi-Router uses Google Vertex for Anthropic models and serves Google's Gemini models through the Gemini provider. See Supported Providers.

Variants and regions

A variant is one way a provider offers a model. Most providers offer a single standard variant; AWS Bedrock offers in_region, geo and global inference scopes; and several providers offer dated snapshots and faster or quantized builds. A region is a location where a variant runs, and the model id and price can differ from one region to another. When a router leaves the region unset, Multi-Router uses the provider key's region on AWS Bedrock, the organization default region on xAI, and global elsewhere, and Google Vertex always uses the location set on the provider key. Multi-Router never moves a request out of the selected region. See Regions & Variants.

The model tuple

The tuple (model, provider, variant, region) identifies one catalog entry: a specific model, served by a specific provider, in a specific variant and region. Each tuple has its own model id, which is the id Multi-Router sends to the provider, and its own price. The following are some of the tuples for anthropic/claude-haiku-4.5:

ProviderVariantRegionModel id sent to the providerInput / output per 1M tokens
anthropicstandardglobalclaude-haiku-4-5$1.00 / $5.00
anthropic20251001globalclaude-haiku-4-5-20251001$1.00 / $5.00
bedrockgeous-east-1us.anthropic.claude-haiku-4-5-20251001-v1:0$1.10 / $5.50
bedrockgeoeu-central-1eu.anthropic.claude-haiku-4-5-20251001-v1:0$1.10 / $5.50
bedrockglobalus-east-1global.anthropic.claude-haiku-4-5-20251001-v1:0$1.00 / $5.00
vertexstandardus-east5claude-haiku-4-5@20251001$1.10 / $5.50

Routers, keys and the catalog

A router connects a request to providers, provider keys and the catalog, and each kind of router does so differently. A static router stores a model, provider and variant and optionally a region, so moving an application to a different provider, variant or region requires a change to the router and no change to the application.

RouterProvider and modelProvider keyCatalog checks
Static routerOne model on one provider, with a variant and an optional region.The router's key, or the organization's default key for the provider.Every request is checked against the catalog entry. A router on a custom endpoint skips the checks.
Fallback routerThose of each member static router, tried in order.Each member's key.Each member's checks.
Weighted routerThose of each member static router, chosen by weight.Each member's key.Each member's checks.
Claude ProxyForwards Claude Code and Claude Desktop traffic in its native format to the route's provider, variant and region.The Claude Proxy's key, or the developer's own Anthropic credential. See Provider Key.Requests are checked against the models the route serves. See Agents & Editors.
Codex ProxyForwards Codex CLI traffic in the OpenAI Responses format to OpenAI.The organization's OpenAI key, or the developer's own OpenAI API key.Requests must name a catalog OpenAI model. See Codex Proxy.
Gemini ProxyForwards Gemini CLI traffic in the Gemini format to Google.The organization's Gemini key.Gemini models only. See Gemini Proxy.
Adaptive RouterServes Chat Completions and Messages with the members switched on in its model group, chosen by its rules.The organization's key for each provider, or a key set on a member.A requested model must be in the group. See Adaptive Router.

A request sent through an alias uses the provider key and catalog checks of the router the alias points to.

Addressing a router

A request selects its router through the model field or the x-mr-router header:

RequestResult
"model": "multi-router/rt_9Hk3Ws7PzQ"The router with that id serves the request.
"model": "multi-router/production"The alias production resolves to the router it points to.
"model": "production"The router or alias with that id serves the request.
x-mr-router: rt_9Hk3Ws7PzQThe header's router serves the request when model names no router. A static router uses the request's model when its Model source is Use request model, and its own model when it is Override with router's model.

A router named in model takes precedence over the header. A disabled router returns router_disabled, and an unknown router returns router_not_found.

The request lifecycle

The request lifecycle is the sequence of checks and calls Multi-Router performs for each Chat Completions or Messages request:

  1. Rate limit. Multi-Router counts the request against the API key's rate limit, and returns 429 rate_limit_exceeded when it is reached.
  2. Body checks. The body must be valid JSON within the endpoint's size limit, with a non-empty list of messages.
  3. Router selection. Multi-Router selects the router as described in Addressing a router.
  4. API key scope. An API key limited to specific routers receives 404 for any other router.
  5. Budget check. A router over its budget returns 429 budget_exhausted with a Retry-After header.
  6. Attempt. For a static router, and for each member a fallback or weighted router tries, Multi-Router applies the router's bound prompt and parameter defaults, translates the request to the provider's API, loads the provider key, checks the request against the catalog entry (model, mode, lifecycle, provider, variant and request features), selects the region, and calls the provider with the entry's model id. It retries rate limits, overloads, temporary server errors, timeouts and connection failures, honoring the provider's retry delay.
  7. Response. Multi-Router converts the provider's response to the format of the request, OpenAI for Chat Completions and Anthropic for Messages, and adds the routing headers.
  8. Cost and Traffic. Multi-Router prices the token usage, counts the cost toward budgets, and records the request in Traffic according to the router's Request Logging settings.

Fallback behavior

Fallback behavior is how a fallback or weighted router handles a failed attempt: failures in the member's configuration, a budget refusal, a catalog check, a timeout, or a rate limit, authentication or server error from the provider move the request to the next member, while an invalid request returns its error at once. See Fallback Router for the full list.

Response headers

Multi-Router adds the following headers to Chat Completions and Messages responses:

HeaderValueSent
x-mr-request-idA unique id for the request, which support uses to trace it.On every response, including errors.
x-mr-correlation-idThe value the request sent in x-mr-correlation-id.When the request sent one.
x-mr-providerThe provider that served the request.On successful responses.
x-mr-upstream-modelThe slug of the model that served the request.On successful responses.
x-mr-upstream-wire-idThe model id sent to the provider.On successful responses.
x-mr-regionThe region the request was sent to.On successful responses from AWS Bedrock and Google Vertex, and from other providers when the model has more than one region.
x-mr-policyHow the request named its router: static, fallback or weighted for that kind of router named in model, and header for a router selected with x-mr-router.On successful responses.
x-mr-attemptsThe number of routers tried: 1 for a static router, and the number of members tried for a fallback or weighted router.On successful responses.
x-mr-composerThe fallback or weighted router the request entered through.On successful responses through a fallback or weighted router.
x-mr-model-deprecatedtrueWhen the model is deprecated.
x-mr-model-replaced-byThe slug of the successor model.When the model is deprecated and the catalog names a successor.
Retry-AfterThe seconds until the budget period resets.On 429 budget_exhausted.

Requests can carry the following headers:

HeaderUse
x-mr-routerSelects a router.
x-mr-session-idGroups requests into a session, shown as Session in Traffic.
x-mr-correlation-idYour own id for the request, shown as Correlation ID in Traffic and returned in the response.

Costs

Multi-Router calculates the cost of each request from the token counts the provider reports and the catalog price of the region the request was served in. Cache reads and writes, long context rates, the fast service tier and web search calls use their own prices, and for Anthropic the region is the one Anthropic reports the request ran in. A custom variant replaces list prices with negotiated prices. Each cost appears in Traffic and counts toward budgets, while the provider bills your account directly at your agreed rates. A streamed response is priced when the stream ends, including a stream the client stops early. See Model Catalog and Custom Models & Pricing.

Traffic

Traffic is the dashboard's record of each request that reaches a router. The router's Request Logging settings control what it keeps: Request metadata (None, Failures only, Anonymous Metadata or Full Metadata) and Request payloads (None, Failures only, Sample or All Payloads). Cost counts toward budgets at every logging level. The columns that relate to providers and models are:

ColumnValue
RouterThe router the request entered through.
TypeThe kind of that router.
AliasThe alias used to reach the router, when there is one.
ModelThe slug of the model that served the request.
VendorThe provider that served the request.
VariantThe variant of the catalog entry.
RegionThe region the request was billed at.
Wire idThe model id sent to the provider.
In/OutThe input and output tokens.
CostThe cost of the request.
AttemptsThe value of x-mr-attempts.
CustomerThe request's user field.

The request detail view also shows Model requested, Model served, Session, Correlation ID, TTFT and Router overhead. See Observability.

Example

This example sends a request to Claude Haiku 4.5 on AWS Bedrock in the United States and reads how it was served:

  1. In LLM Provider Keys, add an AWS Bedrock key with the AWS Region us-east-1 and Access key (SigV4). Its id is llpk_4Vd8Qm2RtL.
  2. Create a static router on Bedrock geo, which returns the router id rt_9Hk3Ws7PzQ.
  3. Send a Chat Completions request to multi-router/rt_9Hk3Ws7PzQ with a correlation id.
  4. Read the response headers, and find the request in Traffic.
Create the routerbash
curl -X POST https://api.multi-router.ai/v1/routers \
  -H "Authorization: Bearer $MR_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "static",
    "name": "haiku-us",
    "routing": {
      "provider": "bedrock",
      "model": "anthropic/claude-haiku-4.5",
      "variant": "geo",
      "region": "us-east-1",
      "provider_key_id": "llpk_4Vd8Qm2RtL"
    },
    "prompt": { "system_message_mode": "merge" }
  }'
Call the routerbash
curl https://api.multi-router.ai/v1/chat/completions \
  -H "Authorization: Bearer $MR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-mr-correlation-id: order-4411" \
  -d '{
    "model": "multi-router/rt_9Hk3Ws7PzQ",
    "messages": [{ "role": "user", "content": "Hello" }]
  }'
Response headershttp
x-mr-request-id: req_8Fq2LmXo1c
x-mr-correlation-id: order-4411
x-mr-provider: bedrock
x-mr-upstream-model: anthropic/claude-haiku-4.5
x-mr-upstream-wire-id: us.anthropic.claude-haiku-4-5-20251001-v1:0
x-mr-region: us-east-1
x-mr-policy: static
x-mr-attempts: 1

The router names the key's region, us-east-1, and the geo variant keeps inference in the United States. In Traffic, the request's row shows Router rt_9Hk3Ws7PzQ, Vendor bedrock, Variant geo, Region us-east-1, the wire id above, the input and output tokens, and a cost at $1.10 per million input tokens and $5.50 per million output tokens.

Engineering · org_7Tq2Wd9KxP

Overview · Traffic

CustomizeSave view
Time:
1h24h7d30d
CustomFilters
Search any id (request/correlation/session)…
Export
Requests
128,403
Success rate
99.4%
5xx errors
142
Tokens
84.2M
Latency · p95
1,240ms
Latency · p99
3,180ms
Cost
$1,284.50
Customers
312
Requests · Count / time128,403
Outcome · 5xx / time142
Latency · p95 / time1,240ms
Cost · Total / time$1,284.50
Requests
Page 125 / page
Columns
DateTimeRequest IDRouterAliasModelVendorIn/OutCostLatencyStatus
2026-09-3014:32:11req_Lw4Tn8Vb2ksupport-bot—anthropic/claude-sonnet-4.5anthropic2,310/512$0.011.94s200
2026-09-3014:32:09req_9Pc1RmYe4srt_9Hk3Ws7PzQ—anthropic/claude-haiku-4.5bedrock860/210$0.0021640ms200
2026-09-3014:32:07req_8Fq2LmXo1crt_9Hk3Ws7PzQ—anthropic/claude-haiku-4.5bedrock1,240/380$0.0035808ms200
req_8Fq2LmXo1c200anthropic/claude-haiku-4.5bedrock
Full Request Payload →
Latency
808ms
Cost
$0.0035
Tokens
1,240 → 380
TTFT
402ms
Attempts
1
Router overhead
1.2ms
Request
Request ID
req_8Fq2LmXo1c
Timestamp
14:32:07
Organization
org_7Tq2Wd9KxP
Token
tok_4Rf8zLm2Qa
Correlation ID
order-4411
Customer
—
Session
—
Peer IP
203.0.113.24
Routing
Alias
—
Router ID
rt_9Hk3Ws7PzQ
Model requested
anthropic/claude-haiku-4.5
Model served
anthropic/claude-haiku-4.5
Vendor
bedrock
Variant
geo
Region
us-east-1
Wire id
us.anthropic.claude-haiku-4-5-20251001-v1:0
Prompt binding
Prompt ID
—
File path
—
Revision
—
Latency
Total
808ms
TTFT
402ms
Request upload
180µs
Router overhead
1.2ms
Upstream
806ms
Router response
240µs
Tokens
Prompt
1,240
Completion
380
Request bytes
6,412
Response bytes
2,186
Cost
Total
$0.0035
Input
$0.0014
Output
$0.0021
Cache read
$0
Cache write
$0
Provider response
Request upload
180µs
Router · overhead
1.2ms
Upstream200
806ms
Router · response
240µs
Total 807.6ms
2026-09-3014:32:04req_3Zr6KdWq8nclaude-code-eng—anthropic/claude-sonnet-4.5vertex5,120/1,034$0.033.21s200
2026-09-3014:31:58req_6Ym2HcPa5tsupport-bot—anthropic/claude-sonnet-4.5anthropic0/0$0212ms429
The request in Traffic, expanded to show the model, vendor, variant, region and wire id that served it, and its cost. Select to enlarge.

The Providers & Models pages cover the following:

PageCovers
Supported ProvidersEach provider's id, key settings, feature support and format translation.
Provider KeyAdding, rotating, transferring, disabling and choosing keys.
Regions & VariantsChoosing where a request runs and which build serves it.
AWS Bedrock, Google VertexCloud account setup, regions and errors for each platform.
Model CatalogModel entries, capabilities, prices, lifecycle and catalog updates.
Model GroupApproved model sets for Claude Proxies and tools policies.
Custom Models & PricingNegotiated prices and models served from your own endpoints.

Routing, Agents & Editors, Observability and Budgets & Spend Control cover routers, Claude Proxies, Traffic and spending limits.

Key ConceptsSupported Providers
Was this helpful?