multi-router.aidocs

Routing

Weighted Router

Split traffic across static routers by weight, for A/B tests and gradual rollouts.

A weighted router splits traffic across its member static routers. On each request it picks a member at random in proportion to the members' weights, so members weighted 90 and 10 receive about 90% and 10% of requests. Changing the weights changes the split at once, without changing clients, which suits A/B tests between models or providers and gradual rollouts of a new model. When the picked member fails, the request moves to another member, as in a fallback router.

Members and weights

A member is a static router in the same organization with a weight, a whole number of 1 or more. The weights do not need to add up to any total: each member's share is its weight divided by the sum of all weights. A weighted router has between 1 and 50 members, each listed once. A member added in the dashboard starts with weight 1, and the router's Members panel shows each member's share as a percentage of the total.

How a member is selected

Multi-Router draws a new random order of the members for every request, weighted by their weights, and tries the members in that order. The first member is picked with probability equal to its share, and when it fails, the request moves to the next member in the same order. Requests are independent, so a caller is not kept on one member across requests. The failures that move a request to the next member are the same as for a fallback router; see Fallback Router.

Headers, settings and Traffic

A weighted router holds its name, description, tags and members, and each request takes its model, provider key, region and settings from the member that serves it. Responses carry x-mr-policy: weighted (or header when selected with x-mr-router), x-mr-composer with the weighted router's id, and x-mr-attempts. In Traffic, the Router column shows the member that served each request, and the Type column shows weighted. When a weighted router is selected with x-mr-router, the request's model is ignored and each member uses its own model.

Creating a weighted router

A weighted router is created on the Routers page:

  1. Select New router, choose Weighted Router under Routers, and select Continue.
  2. Under Configure, enter a Name, an optional Description and Tags, add members with Add Static router, and set each member's weight.
  3. Select Review, check the members and weights, and select Create router.
Create a weighted router that sends 10% of traffic to a new modelbash
curl -X POST https://api.multi-router.ai/v1/routers \
  -H "Authorization: Bearer $MR_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "weighted",
    "name": "sonnet-rollout",
    "routing": {
      "routers": [
        { "router_id": "rt_9Hk3Ws7PzQ", "weight": 90 },
        { "router_id": "rt_5Qa1Mf3KzE", "weight": 10 }
      ]
    }
  }'

To move more traffic to the new model, change each member's weight with PATCH /v1/routers/{id}/members/{key}, where the key is the member's router id, or change the weights in the router's Members panel.

Send 25% to the new modelbash
curl -X PATCH https://api.multi-router.ai/v1/routers/rt_3Vc8Np2HsY/members/rt_5Qa1Mf3KzE \
  -H "Authorization: Bearer $MR_API_KEY" \
  -H 'If-Match: "1"' \
  -H "Content-Type: application/json" \
  -d '{ "weight": 25 }'

Errors

Creating and changing a weighted router returns the following errors, each with status 400:

ErrorCause
invalid_weightA member has no weight, or a weight of 0.
invalid_jsonA weight is not a whole number.
missing_fieldThe members list is missing or empty.
too_many_membersThe router has more than 50 members.
invalid_memberA member is not a static router in the organization.
duplicate_memberA member is listed twice.
member_existsA member add names a router that is already a member.
field_not_supported_for_kindThe body sets a field that does not apply, such as prompt.
Fallback RouterClaude Proxy
Was this helpful?