Observability
Traffic
The Traffic page: views, time ranges, filters, columns, stats, charts and CSV export.
Traffic is the dashboard page that lists every request the organization's routers recorded, newest first. It opens from Traffic in the top navigation, showing the last 24 hours. Views, time ranges and filters narrow the list, a strip of stats and charts summarizes the requests in view, and any request opens in place to show its full detail.
Views
A view is a saved combination of filters, time range, stats and charts. The views rail on the left lists the built-in views and the person's own saved views:
| View | Stats and charts for |
|---|---|
| Overview | Request volume, success, latency and cost across all requests. |
| Latency | Total latency, time to first token and the time spent at the provider. |
| Errors | Error rates and error kinds. |
| Cost | Total cost and its parts. |
| By model | Requests, latency and cost broken down by model. |
| Customers | Usage broken down by the caller's customer id. |
| Features | Requests and cost broken down by feature label, and cost by model. |
A built-in view sets the stats and charts and applies to every request in the time range and filters. Save view stores the current filters, time range, stats and charts as a saved view, which can be renamed or deleted. Changing a saved view marks it Modified, and saving updates it. A saved view keeps the length of its time range, such as the last 7 days, and applies it from the moment it is opened.
Time range
The time range sets which requests the page shows. The Time control offers 1h, 24h, 7d and 30d, and Custom offers these ranges:
| Range | Kind |
|---|---|
| Last 15 minutes | Relative |
| Last 1 hour | Relative |
| Last 6 hours | Relative |
| Last 24 hours | Relative |
| Last 7 days | Relative |
| Last 30 days | Relative |
| Today | Calendar |
| Yesterday | Calendar |
| This week | Calendar |
| Last week | Calendar |
| This month | Calendar |
| Last month | Calendar |
The page shows up to 30 days of requests. It loads when the range, filters or page size change, and a new choice of range brings in the latest requests.
Filters
Filters narrow the requests to those that match every filter set. The Filters menu offers:
| Filter | Matches |
|---|---|
| Model | The model that served the request. |
| Router | The router the request entered through, or the member router that served it, by id. |
| Provider | The provider that served the request. |
| Status | Any, 2xx, 4xx or 5xx. |
| Streaming | Any, Yes or No. |
| Error kind | The error kind exactly as recorded, such as rate_limit_error or timeout. See Request Detail. |
| Customer | The caller's customer id, such as tenant_acme. |
| Session | The session id, exactly as sent in x-mr-session-id. |
| Correlation | The correlation id, exactly as sent in x-mr-correlation-id. |
| Feature | The feature label sent in X-MR-Feature-Id. Exact matches the whole label, and Prefix matches its start, so support/ matches support/triage and support/reply. |
| Machine issuer | The identity provider of a Machine Identity workload. |
| Machine subject | The workload identity, such as a service account. |
| Machine binding | The machine binding the workload matched. |
| Router type | Any, llm, or fallback. |
The search box finds a request by any of its ids: the request id, a correlation id or a session id, matched exactly.
Requests table
The Requests table lists one request per row, 50 per page by default, with 25 and 100 also available, and the page buttons move to the first, previous, next and last page. Selecting a request id opens the request in place. See Request Detail. The Columns button chooses which columns appear, and the choice is remembered in the browser:
| Column | Shown by default | Shows |
|---|---|---|
| Date | Yes | The date of the request. |
| Time | Yes | The time of the request. |
| Request ID | Yes, always | The request id, which opens the request. |
| Router | Yes | The router that served the request. |
| Alias | Yes | The alias the request used, when it used one. |
| Model | Yes | The model that served the request. |
| Vendor | Yes | The provider that served the request. |
| In/Out | Yes | Input and output tokens. |
| Cost | Yes | The cost of the request. |
| Latency | Yes | Total latency. |
| Status | Yes | The status code. |
| Type | No | The router type. |
| Variant | No | The offering variant, such as standard or global. |
| Region | No | The region that served the request. |
| Wire id | No | The provider's own model id. |
| Customer | No | The caller's customer id. |
| Feature | No | The feature label from X-MR-Feature-Id. |
| Identity type | No | The kind of Machine Identity workload, when one made the request. |
| Machine issuer | No | The identity provider of the workload. |
| Machine subject | No | The workload identity. |
| Machine binding | No | The machine binding the workload matched. |
| Attempts | No | How many attempts the request took. |
Stats and charts
The stat strip and chart strip above the table summarize the requests in view. Customize chooses up to 8 stats and 4 charts, in any order, and a chart can be broken down by a dimension, such as latency by model. Latency metrics offer p50, p90, p95, p99, maximum and average; throughput offers average and p95; attempts offer average, maximum and p95. The metrics are:
| Metric | Group |
|---|---|
| Requests | Requests |
| Success % | Outcome |
| Error rate | Outcome |
| 4xx | Outcome |
| 5xx | Outcome |
| Rate-limited | Outcome |
| Retryable | Outcome |
| Recovered % | Outcome |
| Substituted % | Outcome |
| Upstream error rate | Outcome |
| Upstream 5xx | Outcome |
| Total latency | Latency |
| Time to first token | Latency |
| Upstream latency | Latency |
| Time spent in Multi-Router | Latency |
| Total tokens | Tokens |
| Input tokens | Tokens |
| Output tokens | Tokens |
| Cache read tokens | Tokens |
| Cache create tokens | Tokens |
| Cache hit % | Tokens |
| Tokens per request | Tokens |
| Throughput, output tokens per second | Throughput |
| Total cost | Cost |
| Cost per request | Cost |
| Cost per 1,000 tokens | Cost |
| Input, output, cache read and cache write cost | Cost |
| Attempts | Attempts |
| Multi-attempt % | Attempts |
| Distinct models, providers, routers, customers, features, sessions and error kinds | Distinct |
| Busiest model, provider, router, customer, feature and error kind | Busiest |
A chart can break down by one of these dimensions:
| Dimension |
|---|
| Model |
| Provider |
| Router |
| Customer |
| Feature |
| Error kind |
| Session |
| Machine issuer |
| Machine subject |
| Machine binding |
| Router type |
Export
Export downloads the requests that match the current time range and filters as a CSV file named for the range, newest first, with up to 100,000 rows. A larger export ends with the line # truncated at 100000 rows, and a narrower time range or more filters fetch the rest. The file has these columns:
cost_usd_micros is the cost in millionths of a US dollar, so 1500 is $0.0015. router_id is the router the request entered through, and leaf_router_id is the member router that served it. feature_id is the feature label sent in X-MR-Feature-Id. The columns from classifier_type to turn_count are kept for older records and are empty for current requests.