multi-router.aidocs

Self-Hosting

OverviewEnterprise

Running Multi-Router in your own infrastructure, under an enterprise agreement.

Self-hosting runs the whole of Multi-Router, the API, the dashboard and the Portal, inside a company's own cloud account or data center. Requests, prompts, request records and provider keys stay in infrastructure the company controls, and Multi-Router reaches model providers from there. Self-hosting is available under an enterprise agreement, which provides the Multi-Router server and support for installing and running it.

Architecture

A self-hosted installation is made of these parts:

PartRequiredRole
Multi-Router serverYesOne program that serves the API, the dashboard and the Portal on port 8080. It keeps its data in the database, so several copies can run behind a load balancer. See Operations.
DatabaseYesPostgreSQL, or Amazon DynamoDB, holding accounts, routers, keys, prompts, policies and settings.
Secret storeYesAWS Secrets Manager, or a file, holding the database connection and the encryption keys.
Load balancer or reverse proxyYesTerminates TLS and forwards requests to the server.
ClickHouseFor TrafficStores request records for Traffic, request stats and budget reporting.
Payload storageFor payloadsAmazon S3 or a local directory, for request and response payloads.
SMTP serverFor emailSends invitations, Portal links and budget alerts.

What stays the same

A self-hosted installation runs the same server as the hosted service, with the same API, dashboard, Portal, routers, prompts, budgets, single sign-on, user provisioning and machine identity. The rest of this documentation applies, with the addresses replaced by the installation's own.

The other pages in Self-Hosting cover each step:

PageCovers
RequirementsInfrastructure, addresses and network access.
InstallationStarting the server and the installation wizard.
ConfigurationThe Admin pages: addresses, sign-in, email, Traffic, budgets and more.
OperationsUpgrades, scaling, backups, logs and the model catalog.
Budgets APIRequirements
Was this helpful?