Self-Hosting
OverviewEnterprise
Running Multi-Router in your own infrastructure, under an enterprise agreement.
Self-hosting runs the whole of Multi-Router, the API, the dashboard and the Portal, inside a company's own cloud account or data center. Requests, prompts, request records and provider keys stay in infrastructure the company controls, and Multi-Router reaches model providers from there. Self-hosting is available under an enterprise agreement, which provides the Multi-Router server and support for installing and running it.
Architecture
A self-hosted installation is made of these parts:
| Part | Required | Role |
|---|---|---|
| Multi-Router server | Yes | One program that serves the API, the dashboard and the Portal on port 8080. It keeps its data in the database, so several copies can run behind a load balancer. See Operations. |
| Database | Yes | PostgreSQL, or Amazon DynamoDB, holding accounts, routers, keys, prompts, policies and settings. |
| Secret store | Yes | AWS Secrets Manager, or a file, holding the database connection and the encryption keys. |
| Load balancer or reverse proxy | Yes | Terminates TLS and forwards requests to the server. |
| ClickHouse | For Traffic | Stores request records for Traffic, request stats and budget reporting. |
| Payload storage | For payloads | Amazon S3 or a local directory, for request and response payloads. |
| SMTP server | For email | Sends invitations, Portal links and budget alerts. |
What stays the same
A self-hosted installation runs the same server as the hosted service, with the same API, dashboard, Portal, routers, prompts, budgets, single sign-on, user provisioning and machine identity. The rest of this documentation applies, with the addresses replaced by the installation's own.
Related topics
The other pages in Self-Hosting cover each step:
| Page | Covers |
|---|---|
| Requirements | Infrastructure, addresses and network access. |
| Installation | Starting the server and the installation wizard. |
| Configuration | The Admin pages: addresses, sign-in, email, Traffic, budgets and more. |
| Operations | Upgrades, scaling, backups, logs and the model catalog. |