Most workloads call LLMs with a static API key
It is almost 2027, and AI now runs well beyond chat sessions. Build pipelines summarize release notes, review code and write tests. Backend services classify support tickets and draft replies. Batch jobs label data overnight. Each of these workloads calls a model, and each one needs a credential to do it.
The usual credential is an API key stored in a CI secret, an environment variable or a Kubernetes secret. The key never expires and works from anywhere. Over time it gets copied into build logs, container images, local configuration files and other teams' pipelines. When a key leaks, it stays valid until someone notices, and replacing it means updating every system that uses it at the same time.
Most workloads already have an identity
The platforms these workloads run on already prove who they are. GitHub Actions, GitLab and CircleCI issue each pipeline run a signed OpenID Connect token that names the repository and the branch or environment it ran for. Kubernetes clusters on Amazon EKS, Google GKE and Azure AKS issue each pod a signed token for its service account. Google Cloud does the same for its workloads.
These tokens are short-lived, often only minutes, and the platform issues a new one for every run. Nothing has to be stored, and a token that is copied somewhere stops working on its own.
Multi-Router accepts that identity in place of an API key
With machine identity, a workload sends its platform token to Multi-Router in the same header it would use for an API key. Multi-Router verifies the token against the platform that issued it, finds the binding that allows that workload, and routes the request. The workload stores no Multi-Router secret.
An account admin sets this up once for each platform by registering it as an issuer. Multi-Router reads the platform's public configuration and signing keys, recognizes the kind of platform from its address, and generates an audience value. Workloads request their tokens for that audience, so a token issued for another service cannot be used with Multi-Router.
Account Settings
Machine identity
Let applications authenticate with tokens their own platform issues, instead of an API key.
mr-aud_Qx7TmVn2Lr8KpWd4YhBc3sFz9JgNe6Ua1Ro5Ti0XkCopySet this as the token audience in the CI workflow config. Tokens from this issuer are only accepted if they carry it.
- Issuer type
- GitHub Actions
- If the issuer cannot be reached
- Keep serving with cached keys
- Maximum token lifetime
- 7,200 seconds
Authorized organizations
Only authorized organizations can create bindings on this issuer.
org_3Fh8Wq2LdKBindings using this issuer
3 bindings across 1 organization. Read-only — bindings are created and deleted inside each organization.
repo:example-co/billing:ref:refs/heads/mainEngineering · mib_7Hn2Qw5RtXrepo:example-co/billing:environment:productionEngineering · mib_K4vB9mZp3Lrepo:example-co/docs-siteEngineering · mib_P2cW6yNs8DBindings decide which workloads can use which routers
An organization admin decides which workloads may call which routers by adding bindings. A binding names the issuer, the subject and the routers. The subject is the workload's identity as the platform states it, for example:
| Subject | Covers |
|---|---|
repo:example-co/billing:ref:refs/heads/main | Runs of the billing repository on its main branch |
repo:example-co/billing:environment:production | Jobs in the repository's production environment |
system:serviceaccount:payments:api | The api service account in the payments namespace |
system:serviceaccount:payments (prefix) | Every service account in the payments namespace |
A binding can match one exact subject, or every subject under a prefix, so a new service in a namespace can use the namespace's routers without a new binding. When several bindings match, the most specific one applies. A binding gives a workload access to inference and, if chosen, to the list of available models.
Org Settings
Machine identity bindings
Which callers may reach which routers in Engineering, and what they are allowed to do.
Callers connect to https://api.multi-router.ai Copy
repo:example-co/billing:ref:refs/heads/mainexactciGitHub Actions · 1 router · inferencerepo:example-co/docs-siteprefixciGitHub Actions · 2 routers · inferenceEvery request is verified before it is routed
On every request, Multi-Router checks the token's signature against the issuer's published keys, and checks the issuer, the audience, the expiry and how long the token was issued for. Each issuer has a maximum token lifetime, two hours by default, and longer-lived tokens are refused. Multi-Router refreshes each issuer's signing keys every ten minutes and fetches new keys as soon as a token names one it has not seen.
Disabling a binding or an issuer stops its workloads on every Multi-Router server within about a minute. Administrators also choose what happens if an issuer's key endpoint cannot be reached: keep accepting tokens signed with the keys already fetched, or refuse requests until the issuer is reachable again.
- 1 Platform Issues a short-lived token GitHub Actions or the cluster signs a token naming the workload, for the Multi-Router audience.
- 2 Workload Sends it as the bearer token The same Authorization header an API key would use. Nothing is stored.
- 3 Multi-Router Verifies the token Signature against the issuer’s keys, then issuer, audience, expiry and lifetime.
- 4 Multi-Router Matches a binding The issuer and subject select the most specific enabled binding.
- 5 Multi-Router Routes the request To the binding’s router, and records the issuer, subject and binding in Traffic.
A GitHub Actions workflow needs one extra step
A workflow asks GitHub for a token with the issuer's audience and sends it as a bearer token. The workflow has no stored Multi-Router credential:
permissions:
id-token: write
contents: read
jobs:
summarize:
runs-on: ubuntu-latest
steps:
- name: Get a Multi-Router token
run: |
TOKEN=$(curl -s -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=$MR_AUDIENCE" | jq -r .value)
echo "::add-mask::$TOKEN"
echo "MR_TOKEN=$TOKEN" >> "$GITHUB_ENV"
- name: Summarize the release
run: |
curl https://api.multi-router.ai/v1/chat/completions \
-H "Authorization: Bearer $MR_TOKEN" \
-H "Content-Type: application/json" \
-d '{ "messages": [{ "role": "user", "content": "Summarize the changes in this release." }] }' When the binding has one router, the request needs no model or router name.
A Kubernetes pod reads its token from a file
A pod mounts a projected service account token for the issuer's audience. The cluster writes the token to a file and replaces it before it expires, and the application reads the file for each request:
volumes:
- name: mr-token
projected:
sources:
- serviceAccountToken:
path: token
audience: <issuer audience>
expirationSeconds: 3600 curl $MR_BASE_URL/chat/completions \
-H "Authorization: Bearer $(cat /var/run/secrets/multi-router/token)" \
-H "Content-Type: application/json" \
-d '{ "messages": [{ "role": "user", "content": "Classify this payment dispute." }] }' With the OpenAI SDK, the base URL points to Multi-Router and the token file's contents are passed as the API key.
Traffic shows which workload made each request
Every request records the issuer, the subject and the binding that allowed it. Traffic can be filtered by each of them and can show them as columns, so a team can see which repository, pipeline or service account made a request, which model served it and what it cost.
- Request ID
- req_3Vk8Np2HsQ
- Timestamp
- 2026-11-12 09:14:52.087
- Organization
- org_3Fh8Wq2LdK
- API key
- —
- Identity type
- CI
- Machine issuer
- mii_6Gd2Tz8QwM
- Machine subject
- repo:example-co/billing:environment:production
- Machine binding
- mib_K4vB9mZp3L
- Correlation ID
- —
- Customer
- —
- Session
- —
- Peer IP
- 198.51.100.37
- Alias
- —
- Router ID
- rt_8Mc5Jx3VbN
- Model requested
- —
- Model served
- anthropic/claude-sonnet-5
- Vendor
- anthropic
- Variant
- standard
- Region
- global
- Wire id
- claude-sonnet-5
- Prompt ID
- —
- File path
- —
- Revision
- —
Workloads can call LLMs without storing a secret
Machine identity lets CI pipelines and Kubernetes workloads call Multi-Router with the short-lived identity their platform already gives them. There is no long-lived key to store, leak or replace, every request is tied to the workload that made it, and admins control access through bindings. The Machine identity docs cover setup for GitHub Actions, Kubernetes and the other supported platforms.