Budgets & Spend Control
OverviewEnterprise
Capping and tracking spend on AI models, with notifications and blocks at the points you choose.
A budget tracks what an organization spends on AI models over a period and acts at the points you set. Each budget covers part of the organization's traffic, such as everything, a list of routers, or every router with a tag, and holds caps on spend, in total, per router and per model. Rules on the budget decide what happens as spend approaches a cap: an email, a webhook, or blocking requests until the period resets.
Parts of a budget
A budget is made of these parts:
| Part | Purpose |
|---|---|
| Scope | Which traffic the budget covers: the whole organization, a list of routers, or routers with a tag. |
| Period | Daily, weekly or monthly, reset at 00:00 UTC, or lifetime, which never resets. |
| Caps | A total cap, caps per router, and caps per model. |
| Rules | What happens at a point of a cap: notify by email or webhook, or block. |
Tracking and enforcing
A budget's Notify rules track spend and send notifications; a Block rule refuses requests once spend reaches its line, until the period resets or the cap is raised. Multi-Router checks budgets before each request and updates spend every few seconds, so a block takes effect within seconds of the line. Requests already running when the line is crossed finish and are counted, so spend can pass a cap by the cost of those requests.
What counts
Spend is the cost of each inference request, worked out from the model's price as in Traffic. Every request a router serves counts: failed requests count $0, and a stream the caller closed early counts the usage that had arrived. Token counting is free and stays outside budgets. Requests to models with no price count at $0. See Cost.
Several budgets
Every enabled budget that covers a request is checked on its own, and a request is refused when any one of them blocks it. A router in several budgets therefore stops at the lowest line among them. Raising one budget lifts only that budget's block.
Budgets from tools policies
Each tool in a tools policy has a budget, created and managed by the policy, that caps each person's spend on the tool. These budgets appear on the Budgets page as Managed by policy, and their caps and rules change by editing the policy. See Tools Policies.
Related topics
The other pages in Budgets & Spend Control cover each part in detail:
| Page | Covers |
|---|---|
| Creating a Budget | Scopes, caps and periods. |
| Rules and Alerts | Notifications and blocks. |
| Webhooks | Sending budget alerts to your own systems. |
| Overrides and Requests | Raising a cap for one period, and requests for more. |
| Status and Usage | The Budgets page, status labels and usage charts. |
| When a Budget Blocks | What callers receive, on each endpoint. |
| Budgets API | Managing budgets with /v1/budgets. |