multi-router.aidocs

Machine Identity

KubernetesEnterprise

Calling Multi-Router from a pod with a projected service account token.

A pod in a Kubernetes cluster authenticates to Multi-Router with a projected service account token: a token the cluster issues to the pod's service account for a chosen audience, and rotates on its own. The cluster is the issuer, and the service account is the subject. This page uses an EKS cluster; GKE and AKS clusters work the same way with their own issuer URLs.

Registering the cluster

The cluster is registered once, as an issuer:

  1. Find the cluster's OIDC issuer URL. For EKS, it is shown on the cluster's Overview tab as OpenID Connect provider URL.
  2. Register it as an issuer in Multi-Router, and authorize the organization that owns the routers. See Issuers.
  3. Copy the issuer's token audience.

Binding the service account

In the organization, add a binding with Caller type Kubernetes service account and the subject system:serviceaccount:payments:api, for the service account api in the namespace payments, with the router it calls. See Bindings.

Configuring the pod

The pod mounts a projected token for the issuer's audience, with a lifetime at or below the issuer's maximum token lifetime:

Pod specyaml
apiVersion: v1
kind: Pod
metadata:
  name: api
  namespace: payments
spec:
  serviceAccountName: api
  containers:
    - name: api
      image: example-co/payments-api:1.4.0
      env:
        - name: MR_BASE_URL
          value: https://api.multi-router.ai/v1
      volumeMounts:
        - name: mr-token
          mountPath: /var/run/secrets/multi-router
          readOnly: true
  volumes:
    - name: mr-token
      projected:
        sources:
          - serviceAccountToken:
              path: token
              audience: mr-aud_EXAMPLEp3Qk9Zt2Lw8Vd4Nc6Ys1Hb7Rm0Xf5Jg2TaQe
              expirationSeconds: 3600

The application reads the token from the file for each request, since the cluster replaces it before it expires, and sends it as a bearer token:

A request from the podbash
curl $MR_BASE_URL/chat/completions \
  -H "Authorization: Bearer $(cat /var/run/secrets/multi-router/token)" \
  -H "Content-Type: application/json" \
  -d '{ "messages": [{ "role": "user", "content": "Classify this payment dispute." }] }'

With one router on the binding, the request needs no model or router name. With the OpenAI SDK, set the base URL and pass the token file's contents as the API key, read again before it expires.

Every service account in a namespace

A binding with Anything under a prefix and the subject system:serviceaccount:payments covers every service account in the payments namespace, so new services there reach the binding's routers without a new binding.

BindingsGitHub Actions
Was this helpful?