multi-router.aidocs

Observability

Cost

How Multi-Router prices each request, and what each part of the cost covers.

Cost is the price of a request, worked out when the request completes from the tokens it used and the price of the model offering that served it. Multi-Router stores the cost with the request, so Traffic, exports and budgets all show the same figure, and a later price change applies to later requests while earlier requests keep the price they were served at.

Where prices come from

Each model offering in the catalog has prices per million tokens for input, output, cache reads and cache writes, and some offerings add prices for long prompts, faster service tiers and tool use. A request is priced at the offering that served it: the model, provider, variant and region in its record. A custom offering, such as a copy of a catalog offering with a negotiated rate, has its own prices, and a router that uses it has its requests priced at them. See Model Catalog and Custom Models & Pricing.

Parts of the cost

The cost of a request is the sum of these parts:

PartTokens or unitsPrice
InputPrompt tokens, less the cached and cache-write tokensInput price
OutputCompletion tokensOutput price
Cache readPrompt tokens read from the prompt cacheCache read price
Cache writePrompt tokens written to the prompt cache, for each cache lifetimeCache write price for that lifetime
Web searchWeb searches the model ranWeb search fee per search
File searchFile searches the model ranFile search fee per search

Example

This example prices a request that reads most of its prompt from the cache. A request to an offering priced at $3.00 per million input tokens, $15.00 per million output tokens and $0.30 per million cache-read tokens sends 12,000 prompt tokens, of which 10,000 are read from the cache, and receives 800 completion tokens:

PartCalculationCost
Input2,000 × $3.00 / 1,000,000$0.0060
Cache read10,000 × $0.30 / 1,000,000$0.0030
Output800 × $15.00 / 1,000,000$0.0120
Total$0.0210

Long prompts and service tiers

Some offerings charge different rates when a prompt reaches a set number of tokens, counting cached and cache-write tokens, and every token part of the request is then priced at the long-context rates; web and file search fees stay the same. Some providers also offer faster or slower service tiers, such as priority or flex, at their own rates; when the provider reports the tier that served a request, the request is priced at that tier's rates, or at $0 when the offering has no price for that tier.

Unpriced usage

When an offering has no price for a part of the cost, such as a model with no catalog price or a missing cache write price, that part is recorded at $0. The Budgets page shows the share of unpriced traffic as Unpriced traffic. A router can be moved to a custom offering with prices, which prices its later requests.

Where cost appears

The stored cost of each request appears across the dashboard and in exports:

PlaceShows
Traffic Cost columnThe total cost of each request.
Request detailThe total and each part. See Request Detail.
Traffic statsTotal cost, cost per request, cost per 1,000 tokens, and the cost split.
CSV exportcost_usd_micros, in millionths of a US dollar.
Budget usageSpend for each budget over time, by router and by model.
Request LoggingRequest Tracing
Was this helpful?