multi-router.aidocs

Getting Started

Key Concepts

The terms used across the Multi-Router docs, each with where to read more.

Multi-Router's terms fall into four areas: where models come from, how requests are routed, who can do what, and the prompts, records and spend around each request. Each term below links to the page that covers it in full.

Providers and models

Models come from providers, through the organization's keys:

TermMeaning
ProviderA company or platform that serves models, such as Anthropic, OpenAI or AWS Bedrock.
Provider keyThe organization's credential for a provider, used to send it requests.
Model catalogThe list of models Multi-Router knows, with their offerings and prices.
Model tupleThe model, provider, variant and region that identify one way to serve a model.
VariantA provider's way of serving a model, such as standard or geo.
Model groupAn approved set of models, used by Adaptive Routers, Claude Proxies and tools policies.

Routing

Routers decide how each request is served:

TermMeaning
RouterA named configuration that serves requests, holding the model, provider, prompt and settings.
Static routerA router that serves one model from one provider.
Fallback routerA router that tries its member routers in order until one succeeds.
Weighted routerA router that spreads requests across its members by weight.
Claude ProxyA router that serves Claude Code and Claude Desktop in their own format.
Adaptive RouterA router over a model group whose rules choose the model for each request.
Codex ProxyA router that serves Codex CLI in the OpenAI Responses format.
Gemini ProxyA router that serves Gemini CLI in the Gemini format.
AliasA second name that points to a router, and can be repointed.

Access

Accounts, organizations and credentials decide who can do what:

TermMeaning
AccountThe company's top-level space, which is also an organization.
OrganizationA team's space inside the account, with its own resources.
API keyThe credential an application uses to call Multi-Router.
ScopeA permission on an API key to use a group of endpoints.
Tools policyAn organization's rules for the AI tools people set up from the Portal.
Machine identityA workload's platform identity, used in place of an API key.

Prompts, records and spend

Prompts shape requests, Traffic records them, and budgets cap their cost:

TermMeaning
PromptSaved, versioned instructions that a router applies to its requests, combined with the caller's own system message by the router's Prompt composition.
Prompt bindingThe link between a router and its prompt, at the latest or a pinned revision.
TrafficThe record of every request, with its routing, tokens, cost and latency.
BudgetCaps on spend over a period, with notifications and blocks.
Using an SDKOverview
Was this helpful?