API Access
Rate Limit
The number of requests each API key can send per minute, and the response when a key exceeds it.
A rate limit caps how many requests an API key can send in a period of time. Multi-Router applies a rate limit to every key, which protects the organization's provider accounts and budgets from a runaway script or a leaked key. The limit counts requests over a rolling 60-second window, so a key that reaches it can send again as soon as earlier requests fall outside the window.
Limits
Each API key has the following limits:
| Endpoint | Counts toward |
|---|---|
POST /v1/chat/completions | Inference, 300 requests per minute |
POST /v1/messages | Inference, 300 requests per minute |
POST /v1/responses | Inference, 300 requests per minute |
Gemini Proxy endpoints under /v1beta/models | Inference, 300 requests per minute |
| Claude Proxy Bedrock endpoints, including token counting | Inference, 300 requests per minute |
| Claude Proxy Vertex endpoints, other than token counting | Inference, 300 requests per minute |
POST /v1/messages/count_tokens | Token counting, 300 requests per minute |
| Claude Proxy Vertex token counting | Token counting, 300 requests per minute |
Token counting has its own limit, which keeps inference capacity free for requests to models. Each key has one inference count and one token counting count, shared by every endpoint in the row.
A workload that signs in with Machine Identity has the same limits for each machine binding and subject. See Machine Identity.
The 429 response
A request over the limit is refused before it reaches a provider:
On Messages the response uses the Anthropic error format with the same type and message, and on a Claude Proxy's Bedrock and Vertex endpoints it uses their error format with the same message. The OpenAI and Anthropic SDKs retry 429 responses automatically with exponential backoff.
Other limits
Rate limits are one of several controls on usage, each with its own purpose and response:
| Control | Limits | Response |
|---|---|---|
| Rate limit | Requests per key per minute. | 429 rate_limit_exceeded |
| Budget | Spending over a period, for the organization, chosen routers or router tags. | 429 budget_exhausted, with Retry-After |
| Provider rate limit on a static, fallback or weighted router | The provider's own limits on the organization's provider key. | Multi-Router retries, a fallback router moves to its next member, and otherwise the caller receives 429 upstream_rate_limit. |
| Provider rate limit on a Claude Proxy, Codex Proxy or Gemini Proxy | The provider's own limits on the key or login in use. | The provider's 429, passed through as sent. |