Observability
Overview
What Multi-Router records about each request, and where to see usage, latency, errors and cost.
Observability in Multi-Router is the record of every request that passes through a router: who sent it, which router and model served it, how long it took, how many tokens it used, what it cost, and whether it succeeded. The record is kept for each request individually, so a single response can be traced to the exact router, model, prompt revision and API key behind it, and the same records add up to usage, latency, error and cost figures for the organization.
What is recorded
Each request record holds the following fields:
| Group | Field |
|---|---|
| Request | Request id |
| Request | Time |
| Request | Organization |
| Request | API key id |
| Request | Caller's customer id |
| Request | Caller's session id |
| Request | Caller's correlation id |
| Request | Peer IP |
| Routing | Alias used |
| Routing | Router |
| Routing | Model requested |
| Routing | Model served |
| Routing | Provider, variant and region |
| Routing | Provider's model id |
| Prompt | Bound prompt and the revision or commit used |
| Outcome | Status code |
| Outcome | Error kind and error message |
| Outcome | Attempts |
| Tokens | Input and output tokens |
| Tokens | Cached and cache-write tokens |
| Tokens | Request and response sizes |
| Cost | Total cost and its parts |
| Latency | Total time and time to first token |
| Latency | Time at the provider and time spent in Multi-Router |
| Machine identity | Issuer, subject and machine binding of a Machine Identity workload |
Routers can also store the full request and response, called payloads, for some or all requests. See Request Logging.
Where to see it
Request records are shown on several dashboard pages, each for a different purpose:
| Place | Shows |
|---|---|
| Traffic | Every request, with filters, views, stats, charts and CSV export. |
| Request Detail | One request in full, opened from Traffic, with its payloads when they were stored. |
| Routers page | Requests, errors and p95 latency over the last 24 hours for the organization, and requests over the last 24 hours for each router. |
| Budget usage | Spend over time, by router and by model, for each budget. See Budgets & Spend Control. |
What decides what is recorded
Each router's Request Logging settings decide how much it records: full request metadata, metadata without fields that identify end users, failed requests only, or nothing, and whether payloads are stored. An administrator's Storage Policy sets the most any router can record. See Request Logging.
Cost
Multi-Router prices each request when it completes, from the price of the model offering that served it: a catalog offering, or a custom offering with its own prices. The cost is stored with the request, so totals in Traffic and budgets reflect the prices at the time of each request. See Cost.
Tracing a request
Every response carries an x-mr-request-id header with the request's id, which finds the request in Traffic. Callers can also send their own session and correlation ids, which Traffic records and searches, to tie requests to their own logs. See Request Tracing.
Related topics
The other pages in Observability cover each part in detail:
| Page | Covers |
|---|---|
| Traffic | Views, filters, columns, stats, charts and export. |
| Request Detail | Every field of a request, errors, and payloads. |
| Request Logging | What each router records, and the Storage Policy. |
| Cost | How each request is priced. |
| Request Tracing | Request, session and correlation ids, and response headers. |