The Ultimate AI Gateway
One API across every model and provider, with the spend controls, governance, and visibility to run AI in production.
Designed for Performance at Scale
Overview · Traffic
| Time | Request ID | Router | Alias | Model | Customer | In/Out | Cost | Lat | Att | Status |
|---|---|---|---|---|---|---|---|---|---|---|
| 07:32:14.291 | req_8f2a91 | rt_chatbot | smart | claude-sonnet-4-6 | tenant_acme | 1,204 / 856 | $0.0142 | 842ms | 1 | 200 |
| 07:32:13.884 | req_7d10c4 | rt_agent | — | gpt-5 | user_42891 | 2,860 / 1,420 | $0.0388 | 1,910ms | 1 | 200 |
| 07:32:12.530 | req_5c93af | rt_summarize | fast | claude-haiku-4-5 | tenant_globex | 640 / 220 | $0.0021 | 318ms | 1 | 200 |
| 07:32:10.119 | req_4b77e2 | rt_agent | smart | gpt-5 | tenant_initech | 3,120 / 980 | $0.0402 | 2,240ms | 2 | 200 |
| 07:32:08.742 | req_3a18bd | rt_chatbot | cheap | gemini-2.5-pro | user_10233 | 1,540 / 612 | $0.0096 | 1,120ms | 1 | 200 |
| 07:32:06.501 | req_29ee70 | rt_embed | — | text-embedding-3-large | tenant_acme | 820 / 0 | $0.0003 | 92ms | 1 | 200 |
| 07:32:04.330 | req_1f0c55 | rt_agent | smart | gpt-5 | tenant_hooli | — | — | — | 3 | 429 |
| 07:32:01.987 | req_0b6d18 | rt_chatbot | fast | claude-sonnet-4-6 | user_88120 | — | — | 1,480ms | 2 | 503 |
The platform for running AI in production
Routing, prompts, cost controls, and observability in one platform.
Low-overhead gateway
A Rust gateway with full streaming and per-request latency tracking.
OpenAI and Anthropic compatible API
Drop-in endpoints compatible with the OpenAI and Anthropic APIs, across 16 providers.
Routing and fallbacks
Compose routers with aliases, automatic fallbacks, and weighted load balancing.
Prompt management
Version prompts inline or sync them from GitHub, with {{variables}} and full revision history.
Traffic observability
Latency, errors, cost, and every request, filterable and saved as views.
Cost and usage tracking
Per-request cost and token accounting, cache-aware, attributed by customer.
Every provider, every region
Routers reach every Bedrock, Vertex and Azure region and every major model API. Select a region to see the models it serves and what they cost there.