Providers & Models
Supported Providers
The model providers Multi-Router routes to, the settings each provider key needs, the request features each provider supports, and how requests and responses are translated.
Multi-Router routes requests to several kinds of model provider. Model publishers such as Anthropic, OpenAI and Google serve their own models. Inference platforms such as Fireworks, Groq and Together serve open models. Cloud platforms such as AWS Bedrock and Google Vertex serve models from many publishers across regions. Multi-Router also routes to custom endpoints that implement the OpenAI Chat Completions API. Your application calls every provider through the same two endpoints, Chat Completions and Messages, and Multi-Router translates each request into the provider's own API and each response back.
Providers
A provider is a company or cloud platform that serves models to Multi-Router, identified by a provider id that router configurations use and the x-mr-provider response header returns. A router whose provider does not serve its model returns provider_not_offered.
| Provider | Provider id | Publishers | Provider key settings |
|---|---|---|---|
| Anthropic | anthropic | Anthropic | API key |
| OpenAI | openai | OpenAI | API key and Region (data residency, default Global) |
| Google Gemini | gemini | API key | |
| Cohere | cohere | Cohere | API key |
| AWS Bedrock | bedrock | Amazon, Anthropic, DeepSeek, Meta, Mistral, Moonshot, Nvidia, OpenAI, Qwen, Z.ai | AWS Region, and an access key (SigV4) or Bedrock API key. See AWS Bedrock. |
| Google Vertex | vertex | Anthropic | Service account, project and location. See Google Vertex. |
| DeepSeek | deepseek | DeepSeek | API key |
| Fireworks | fireworks | DeepSeek, Moonshot, OpenAI, Z.ai | API key |
| Groq | groq | Moonshot, OpenAI | API key |
| Mistral | mistral | Mistral | API key |
| Together | together | DeepSeek, Moonshot, OpenAI, Z.ai | API key |
| xAI | xai | xAI | API key |
| Z.ai | zai | Z.ai | API key |
| Custom endpoint | custom | Any model the endpoint serves | Base URL, API format and API key. See Custom endpoints below. |
Provider endpoints
The provider endpoints are the hosts Multi-Router calls for each provider, with the authentication it uses and the regions it serves. A self-hosted installation allows outbound HTTPS on port 443 to the hosts of the providers it uses.
| Provider | Upstream endpoint | Authentication | Regions |
|---|---|---|---|
| Anthropic | https://api.anthropic.com/v1/messages | x-api-key | global, us |
| OpenAI | https://api.openai.com/v1, and https://us.api.openai.com/v1, https://eu.api.openai.com/v1 or https://ae.api.openai.com/v1 for data residency | Bearer token | global, us, eu, ae |
| Google Gemini | https://generativelanguage.googleapis.com/v1beta | x-goog-api-key | global |
| Cohere | https://api.cohere.com/v2/chat | Bearer token | global |
| AWS Bedrock | https://bedrock-runtime.{region}.amazonaws.com | AWS SigV4 or Bedrock API key | AWS regions |
| Google Vertex | aiplatform.googleapis.com, aiplatform.us.rep.googleapis.com, aiplatform.eu.rep.googleapis.com and {region}-aiplatform.googleapis.com, plus oauth2.googleapis.com | Access token from the service account | Google Cloud locations |
| DeepSeek | https://api.deepseek.com/v1 | Bearer token | global |
| Fireworks | https://api.fireworks.ai/inference/v1 | Bearer token | global |
| Groq | https://api.groq.com/openai/v1 | Bearer token | global |
| Mistral | https://api.mistral.ai/v1 | Bearer token | global |
| Together | https://api.together.xyz/v1 | Bearer token | global |
| xAI | https://api.x.ai/v1 | Bearer token | us-east-1, us-west-2, eu-west-1 |
| Z.ai | https://open.bigmodel.cn/api/paas/v4 | Bearer token | global |
| Custom endpoint | {base URL}/chat/completions | Bearer token | Set by the custom model |
DeepSeek, Fireworks, Groq, Mistral, Together, xAI, Z.ai and custom endpoints use the OpenAI Chat Completions format, and Multi-Router sends them the same request it sends OpenAI, with max_tokens sent as max_completion_tokens. Regions are described in Regions & Variants.
Provider-specific behavior
| Provider | Behavior |
|---|---|
| Anthropic | System and developer messages anywhere in the conversation become the system prompt, and tool results are sent as user messages. Claude requests on Google Vertex and AWS Bedrock follow the same behavior. |
| Google Gemini | System and developer messages are combined into Gemini's system instruction. Gemini returns no tool call id, so Multi-Router uses the function name as the id, and a tool result is matched to its call by function name. |
| Cohere | Cohere receives the text of each message. A json_schema response format is sent as Cohere's JSON mode with the schema, and top_p and top_k are sent as p and k. |
| AWS Bedrock | Image URLs are downloaded and sent to Bedrock as image data. See AWS Bedrock. |
| Google Vertex | Serves Anthropic models only, and requests for other publishers' models return publisher_not_supported. See Google Vertex. |
| xAI | The region is a label for the request: every region of a model has the same price and is served from the same endpoint. |
Custom endpoints
The custom provider sends requests to an endpoint that implements the OpenAI Chat Completions API, such as a self-hosted model server. Its provider key holds the Base URL, the API format (OpenAI Chat Completions) and the API key, and Multi-Router appends /chat/completions to the base URL, which must use HTTPS and a public DNS name, with no port, credentials, query string or fragment. A router on a custom endpoint can name any model the endpoint serves, and requests skip the catalog checks. Requests carry no price unless the model is registered with a price in Custom Models & Pricing.
Feature support
Feature support shows which request features each provider supports: streaming responses, tool calls, parallel tool calls, image input, structured output that follows a JSON schema, reasoning, and prompt caching. Per model means some of the provider's models support the feature, and the Models page lists the features of each model. A request that uses a feature its model lacks returns an error named for the feature; see Model Catalog.
| Provider | Streaming | Tools | Parallel tool calls | Images | Structured output | Reasoning | Prompt caching |
|---|---|---|---|---|---|---|---|
| Anthropic | Yes | Yes | Yes | Yes | Claude 4.5 and later | Claude 3.7 and later | Yes |
| Google Vertex | Yes | Yes | Yes | Base64 only | Claude 4.5 and later | Claude 3.7 and later | Yes |
| AWS Bedrock | Per model | Per model | Claude models | Per model | No | Claude 3.7 and later | Claude models |
| OpenAI | Per model | Per model | Per model | Per model | Per model | Per model | Per model |
| Google Gemini | Per model | Yes | No | Yes | Per model | Per model | Per model |
| Cohere | Yes | Yes | No | Per model | Yes | Per model | No |
| DeepSeek | Yes | Yes | No | No | Yes | Yes | Yes |
| Fireworks | Yes | Yes | No | Per model | Yes | Per model | Per model |
| Groq | Yes | Yes | No | No | Yes | Per model | No |
| Mistral | Yes | Yes | No | Per model | Yes | No | No |
| Together | Yes | Yes | No | Per model | Yes | Per model | Per model |
| xAI | Yes | Yes | Yes | Yes | Yes | Per model | Yes |
| Z.ai | Yes | Per model | No | Per model | Per model | Per model | Per model |
Claude features by endpoint
Claude models on Anthropic, Vertex and Bedrock support some features on only one of the two endpoints, as follows:
| Feature | /v1/chat/completions | /v1/messages |
|---|---|---|
| Structured output | response_format with json_schema, on Anthropic and Vertex | Not available |
| Reasoning | Not available | thinking with type: "enabled" and budget_tokens, returned as thinking blocks |
| Prompt caching | Not available | cache_control on content blocks, system blocks and tools |
| Maximum output tokens | max_tokens or max_completion_tokens; 1,024 when neither the request nor the router sets one | max_tokens is required |
Input content
Input content is the set of content types a request message can contain. Requests can contain text, images, tool calls and tool results; images are sent as a URL or, on Chat Completions, as a data: URL with base64 data. Google Vertex accepts base64 images only, AWS Bedrock receives URL images after Multi-Router downloads them, and Cohere receives text only. On Chat Completions, a tool message keeps its text parts, and a content part of any type other than text and image returns not_supported. The Messages endpoint also accepts thinking blocks, and removes other block types, such as redacted_thinking and search result blocks, before sending the request.
Translation between API formats
Translation is the conversion of a request into the provider's API and of the response back into the format of the request. Multi-Router accepts OpenAI Chat Completions requests at /v1/chat/completions and Anthropic Messages requests at /v1/messages, and either format can target any provider. Messages, system prompts, text, tools, tool choice, tool calls and results, stop sequences, temperature, top_p and maximum tokens are converted for every provider. Other fields are converted when the provider's API has an equivalent and removed otherwise, and fields Multi-Router does not recognize are removed.
| Endpoint | Provider | Fields removed |
|---|---|---|
| Chat Completions | OpenAI | Image detail. Every other recognized field is sent. |
| Chat Completions | OpenAI-compatible providers | Image detail. Every other recognized field is sent. |
| Chat Completions | Claude models | reasoning_effort, seed, logprobs, top_logprobs, penalties, logit_bias, parallel_tool_calls, prompt_cache_key, prompt_cache_retention, service_tier, store, metadata, user, safety_identifier, verbosity, prediction, tool strict, image detail |
| Chat Completions | Google Gemini | reasoning_effort, prompt cache fields, parallel_tool_calls, penalties, logprobs, top_logprobs, logit_bias, service_tier, store, metadata, user, safety_identifier, verbosity, prediction, tool strict, non-text tool results |
| Chat Completions | Cohere | Images, reasoning_effort, prompt cache fields, parallel_tool_calls, logprobs, top_logprobs, logit_bias, service_tier, store, metadata, user, safety_identifier, verbosity, prediction, tool strict |
| Chat Completions | Non-Claude models on Bedrock | Every field outside the converted set. |
| Messages | Every provider | output_config, service_tier, container, mcp_servers, context_management, disable_parallel_tool_use |
| Messages | OpenAI | top_k, thinking and thinking blocks, cache_control, metadata, non-text tool result content. A tool result with is_error is sent with a [Tool Error] prefix. |
| Messages | OpenAI-compatible providers | top_k, thinking and thinking blocks, cache_control, metadata, non-text tool result content. A tool result with is_error is sent with a [Tool Error] prefix. |
| Messages | Google Gemini | thinking and thinking blocks, cache_control, non-text tool result content |
| Messages | Cohere | thinking and thinking blocks, cache_control, non-text tool result content, images |
The anthropic-beta request header is forwarded to Anthropic and Google Vertex. Reasoning text from the model is omitted from Chat Completions responses. Tool call ids pass through unchanged in both directions, so a tool result names the id the provider issued: toolu_ ids from Claude, call_ ids from OpenAI, and the function name from Gemini.
The following requests return an error:
| Request | Error |
|---|---|
Chat Completions with n greater than 1 | not_supported |
Chat Completions with a name on a message | not_supported |
| Chat Completions with a tool other than a function | not_supported |
| Chat Completions with a content part other than text and image | not_supported |
A json_object or json_schema response format sent to a Claude model before 4.5 | structured_output_not_supported |
A json_object response format sent to Claude 4.5 or later | invalid_request |
Any response_format sent to Claude on AWS Bedrock | invalid_request |
Messages with thinking enabled and no budget_tokens | invalid_request_error |
Messages with a system role inside messages | invalid_request_error |
Streaming
Streaming returns the response as server-sent events in the format of the endpoint, whatever format the provider streams in. Multi-Router always asks the provider for token usage, so Traffic records token counts whether or not the request asks for them.
| Chat Completions | Messages | |
|---|---|---|
| Events | data: {chat.completion.chunk} lines, ending with data: [DONE] | message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop, and ping |
| Keepalive | A : keepalive comment after 15 seconds without data | The same |
| Usage | A final chunk with empty choices and usage, when the request sets stream_options.include_usage | message_delta.usage |
| Tool calls | The first chunk of each call carries id, name and index, and later chunks carry argument fragments | content_block_start with tool_use, then input_json_delta |
| Reasoning | Omitted | thinking_delta and signature_delta |
| Errors after the stream starts | data: {"error": ...}, then data: [DONE] | event: error |
Token usage
Token usage is the count of tokens the provider reports, returned in the format of the endpoint:
| Chat Completions | Messages |
|---|---|
prompt_tokens | input_tokens |
| Input tokens as the provider reports them. Claude providers exclude cache reads and writes, and OpenAI, OpenAI-compatible providers and Gemini include cached tokens. | |
completion_tokens | output_tokens |
| Output tokens, including reasoning tokens. | |
total_tokens | Not returned |
| Input and output tokens combined. | |
prompt_tokens_details.cached_tokens | cache_read_input_tokens |
| Tokens read from the prompt cache. | |
| Not returned | cache_creation_input_tokens |
| Tokens written to the prompt cache. | |
| Not returned | cache_creation |
| Tokens written to the prompt cache, broken down by cache lifetime. | |
completion_tokens_details.reasoning_tokens | Not returned |
| Reasoning tokens, when the provider reports them. | |
| Not returned | inference_geo |
| The region Anthropic reports the request ran in. | |
Response fields
Response fields follow the format of the endpoint, with the following values set by Multi-Router. The model field returns the value the request sent, such as the router name, and the x-mr-upstream-model and x-mr-upstream-wire-id headers name the model that served it. The id is the provider's response id, prefixed with chatcmpl- on Chat Completions. created is the time Multi-Router built the response, and Messages responses return stop_sequence as null. Stop reasons are mapped as follows:
| Provider reason | Chat Completions finish_reason | Messages stop_reason |
|---|---|---|
end_turn | stop, or tool_calls when the response has tool calls | end_turn |
stop | stop, or tool_calls when the response has tool calls | end_turn |
complete | stop, or tool_calls when the response has tool calls | end_turn |
stop_sequence | stop | stop_sequence |
max_tokens | length | max_tokens |
length | length | max_tokens |
tool_use | tool_calls | tool_use |
tool_calls | tool_calls | tool_use |
content_filter | content_filter | end_turn |
safety | content_filter | end_turn |
| Any other reason, such as refusal | stop | end_turn |
Provider errors and retries
A provider error is a failure the provider returns after Multi-Router sends the request. Multi-Router returns it with a Multi-Router error code and an upstream object holding the provider, the provider's error code and its HTTP status; the provider's own message is not passed through, because it can contain account details.
| Error | Status | Provider responses |
|---|---|---|
upstream_invalid_request | 400 | The provider rejected the request as invalid. |
upstream_context_length | 400 | The prompt exceeds the model's context window. |
upstream_content_filter | 400 | The provider's content policy blocked the request. |
upstream_model_not_found | 400 | The provider does not offer the model to the account. |
upstream_invalid_credentials | 502 | The provider rejected the provider key. |
upstream_insufficient_quota | 402 | The provider account has no remaining credit or quota. |
upstream_rate_limit | 429 | The provider rate limited the request or timed out, and retries ran out. |
upstream_server_error | 502 | The provider returned a server error or was overloaded (5xx or 529), and retries ran out. |
upstream_transport | 503 | The connection failed while the response was being read. |
upstream_other | The provider's status | Any other provider error. |
On the Messages endpoint, the same failures use Anthropic error types chosen by status: invalid_request_error for 400, rate_limit_error for 429, overloaded_error for 503 and api_error otherwise, with the upstream object included.
Multi-Router retries a provider request that fails with HTTP 429, 500, 502, 503, 504 or 529, a timeout or a connection failure, for up to five attempts. Waits double from one second with up to 25% random jitter, up to 60 seconds, and a whole-second Retry-After from the provider replaces the computed wait. Retries stop when another attempt would run past five minutes from the first, and a 429 that reports exhausted credit is returned at once. These rules apply to every provider and to custom endpoints, and Claude Proxy requests are sent once, without retries.
Limits
Limits are the size and time bounds Multi-Router applies to provider requests:
| Limit | Value | When exceeded |
|---|---|---|
| Chat Completions request body | 4 MiB | 413 body_too_large |
| Messages request body | 32 MB | 413 request_too_large |
| Time to the provider's response headers, per attempt | 120 seconds | The attempt is retried. Long generations use streaming. |
| Total time across retries | 5 minutes | Retries stop. |
Claude Code and Claude Desktop
A Claude Proxy forwards requests from Claude Code and Claude Desktop to Anthropic, AWS Bedrock or Google Vertex in their native API formats, replacing only the model id, so every feature of those APIs is available. See Agents & Editors.