ManyLayers is distributed as a release bundle — you do not need source code access to install or operate it. Each release includes:
  • Container images — delivered through the ManyLayers private registry (pull credentials provided during onboarding) or as an offline image archive (manylayers-images-<version>.tar) for air-gapped environments.
  • Deployment assets — the infra/docker/ compose stacks (dev, demo, prod) with their .env.<stack>.example templates, and a Helm chart (manylayers-<version>.tgz).
  • A license file — a signed license.json issued for your seat count and term. See On-prem license.
Your ManyLayers account contact provides release bundles, registry credentials, and license files. Contact support if you have not received onboarding credentials.

​
Prerequisites

Confirm your environment meets the requirements. At minimum you need Docker Engine 24+ with Compose v2, or a Kubernetes 1.27+ cluster with Helm 3 for the Kubernetes path.

​
Load the container images

docker login registry.manylayers.ai -u <customer-id>
# password: the pull token from your onboarding email
The compose files and Helm chart reference registry.manylayers.ai/manylayers/* images and pull them automatically on first start.

​
Docker Compose installation

The recommended single-host installation. Three stacks ship in the bundle, one per environment, each a single compose file with a single env file beside it — so there is no way to end up running two databases for one deployment:
StackCompose fileEnv fileWhat it is
prodinfra/docker/docker-compose.prod.yml.env.prodInstall this one. Registry images, read-only root filesystems, resource limits, data stores bound to 127.0.0.1
demoinfra/docker/docker-compose.demo.yml.env.demoAn evaluation stack: baked credentials and a local mock upstream, so it runs with no provider account at all
devinfra/docker/docker-compose.dev.yml.env.devBuilds from source. For contributors, not deployments
Each env file is read twice by compose: --env-file resolves ${VAR} inside the YAML, and env_file: injects the same values into every container. One file, one truth.
1

Set the environment

cp infra/docker/.env.prod.example infra/docker/.env.prod
openssl rand -hex 32     # generate each key and password
Set MANYLAYERS_ADMIN_KEY, POSTGRES_PASSWORD and REDIS_PASSWORD — compose refuses to start without them — and repeat the two passwords inside DATABASE_URL and REDIS_URL. A .env file has no variable expansion, so those URLs carry the credential literally; rotate a password and you must change it in both places.
2

Configure the gateway

The stack mounts configs/gateway.container.yaml read-only at /etc/manylayers/gateway.yaml. Edit it for non-secret settings — routing, cache, audit, PII, model list.Two rules govern that file: it holds no secrets, and no per-deployment values. database.url, redis.url and the listen port are ${DATABASE_URL}, ${REDIS_URL} and GATEWAY_PORT, all set from infra/docker/.env.prod, so one image runs in every environment. Credentials stay ${ENV_VAR} references resolved from the process environment — put their values in infra/docker/.env.prod:
  • MANYLAYERS_CONNECTOR_KEY — passphrase for connector credential storage
  • provider keys such as OPENAI_API_KEY and ANTHROPIC_API_KEY
  • the path to your mounted license.json
3

Start the stack

PROD="docker compose --env-file infra/docker/.env.prod -f infra/docker/docker-compose.prod.yml"

$PROD pull            # or `$PROD build` to build from source
$PROD run --rm migrate
$PROD up -d
From a source checkout the same three steps are make prod-pull, make prod-migrate, make prod-up — or make prod-deploy for all of them plus a readiness check.Migrations are not left to chance: the one-shot migrate service applies the Goose set and exits, and every service waits for it to exit successfully. No long-running binary migrates on startup, so several gateway replicas booting together cannot race each other.The gateway listens on 8180 and is the only service published on all interfaces. The workspace serves the admin dashboard at http://<host>:8190/ui/ and the deployer runs on 8200; both bind 127.0.0.1, so reach them through your ingress or a VPN.
4

Redeploy later

$PROD pull
$PROD up -d --no-deps gateway workspace deployer
(make prod-upgrade.) The data stores are left alone. Scale the data plane with --scale gateway=3, or make prod-scale N=3.

​
Evaluating with the demo stack

To try ManyLayers before wiring in a provider, start the demo stack instead. It pulls :demo images and routes every logical model at a bundled mock upstream, so it needs no provider account, no API key and no outbound network: 
docker compose --env-file infra/docker/.env.demo \
  -f infra/docker/docker-compose.demo.yml up -d
(make up-demo.) Its credentials are baked into the committed template by design — never expose the demo stack publicly.

​
Optional infrastructure services

These are in the dev stack behind the optional profile. To run one without the rest of the stack, name it on the command line — that starts the service even though it is behind a profile: 
docker compose --env-file infra/docker/.env.dev -f infra/docker/docker-compose.dev.yml up -d qdrant
ServiceHost portUse case
redis6479Shared rate limits, budgets and cache. Always started; never behind a profile
kafka9192High-throughput queueing for queue.backend: kafka
qdrant6433Vector store for knowledge bases and semantic caching
minio9100 / 9101S3-compatible storage for model artifacts
dstack3100Managed model deployments via dstack (starts its own database too)

​
Production checklist

  • TLS termination via a reverse proxy (nginx, Caddy, or your load balancer)
  • Postgres backed by durable storage with scheduled backups
  • MANYLAYERS_ADMIN_KEY stored in your secret manager, not in shell history
  • License file mounted read-only into the gateway container

​
Kubernetes (Helm) installation

1

Create a namespace and secrets

kubectl create namespace manylayers

kubectl create secret generic manylayers-keys \
  --namespace manylayers \
  --from-literal=MANYLAYERS_ADMIN_KEY=your-admin-key

kubectl create secret generic manylayers-license \
  --namespace manylayers \
  --from-file=license.json
2

Install the chart from the bundle

helm install manylayers manylayers-<version>.tgz \
  --namespace manylayers \
  --set envFromSecrets[0]=manylayers-keys
3

Enable optional components

helm upgrade manylayers manylayers-<version>.tgz \
  --namespace manylayers \
  --set qdrant.enabled=true \
  --set config.rag.enabled=true \
  --set config.rag.vector_store=qdrant \
  --set config.suite.enabled=true \
  --set deployer.enabled=true

​
Key Helm values

ValueDefaultDescription
replicaCount1Set >1 with Redis configured for shared rate-limit state
image.registryregistry.manylayers.aiOverride for internal image mirrors
qdrant.enabledfalseDeploy an in-cluster Qdrant instance
kafka.enabledfalseEnable the Kafka queue backend
deployer.enabledfalseEnable managed model deployments
deployer.modekuberneteskubernetes or dstack
Set config.database.url to your Postgres connection string and config.redis.addr to share rate-limit and budget state across replicas.
The full set of values is documented in the values.yaml file inside the chart archive.

​
Upgrading

  1. Load the new release’s images (registry pull or docker load).
  2. Re-run the migration job — migrations are forward-only and safe to apply before restarting.
  3. Restart the gateway with the new image tag (docker compose up -d or helm upgrade).

​
Next steps