ManyLayers gives your organization an enterprise-grade AI platform that runs entirely on your own infrastructure. Deploy it once and your team gets an OpenAI-compatible AI gateway, a full-featured workspace for chat and RAG, and managed model deployment and training — all under your control.

AI Gateway

OpenAI-compatible proxy with routing, guardrails, PII firewall, caching, budgets, rate limits, and per-key controls — keeping your AI traffic secure and observable.

Workspace

A complete AI workspace for your team: chat with RAG and knowledge bases, 23+ connectors, web search, voice, agents, workflows, and model evaluations.

Deployment & Training

Deploy open models to your Kubernetes cluster or dstack environment. Run LoRA fine-tuning jobs against your own data, entirely on your infrastructure.

​
Organizations

Every tenant is an organization, and organizations are fully isolated from one another: separate teams, users, API keys, conversations and knowledge bases. Self-serve signup, invite flows and the operator console for managing all orgs are always available — there is nothing to turn on. This is independent of where you run it. A single-team deployment on your own infrastructure is one organization, and works the same way. The mode: onprem / mode: saas config key that used to select between two models has been removed; see Deployment model.

​
How it fits into your stack

​
Key capabilities

  • Drop-in OpenAI compatibility — point your existing OpenAI SDK clients at ManyLayers with a single base URL change. No code rewrites.
  • 18 provider translations — route to Anthropic, Azure, Gemini, Vertex, Bedrock, Cohere, and 12 more. Your clients always use the standard OpenAI format.
  • Settings live in the database — your YAML config seeds initial values; the admin UI overrides them permanently without a restart.
  • All data stays on your infrastructure — no telemetry, no external calls except to the AI providers you configure.

​
Next steps

Requirements

Check what your environment needs before installing.

Quickstart

Get a working gateway running in under 5 minutes.