GET /v1/realtime tunnels a WebSocket session to an OpenAI or Azure OpenAI realtime model. Your client authenticates with a ManyLayers credential; the gateway checks access and limits, attaches the provider credential, and relays the session.

​
How it works

  1. The gateway reads model from the query string and checks that your team may use it.
  2. It resolves the model from the configured catalog (gateway.yaml) and checks the provider is openai or azure.
  3. It evaluates your access policies, model restrictions, rate limits and budgets.
  4. It dials the provider, sends the upgrade with the provider’s own credential (your gateway key is never forwarded), and relays the provider’s handshake response.
  5. After the upgrade, frames flow in both directions unmodified until either side closes.

​
Configuration structure

Realtime models are ordinary gateway.yaml models (self-hosted configuration) whose first upstream is openai or azure. Only the host of url is used; the gateway builds the realtime path itself. 
models:
  - logical_name: gpt-realtime          # the name clients pass as ?model=
    upstreams:
      - provider: openai
        url: https://api.openai.com
        api_key: ${OPENAI_API_KEY}
        model: gpt-4o-realtime-preview  # upstream model name
  - logical_name: azure-realtime
    upstreams:
      - provider: azure
        url: https://my-resource.openai.azure.com
        api_key: ${AZURE_OPENAI_KEY}
        model: gpt-4o-realtime          # Azure deployment name
        api_version: 2024-10-01-preview # default when omitted

​
Key fields

  • ?model=: Required. The gateway model name. Missing returns 400 missing_model.
  • Upstream model: Sent to OpenAI as ?model=, or to Azure as ?deployment= on /openai/realtime. Defaults to the gateway model name.
  • api_version (Azure): Defaults to 2024-10-01-preview.

​
Connect

import asyncio, json, websockets

async def main():
    async with websockets.connect(
        "wss://app.manylayers.io/v1/realtime?model=gpt-realtime",
        additional_headers={"Authorization": "Bearer ml-..."},
    ) as ws:
        await ws.send(json.dumps({
            "type": "response.create",
            "response": {"modalities": ["text"], "instructions": "Say hello."},
        }))
        async for message in ws:
            print(json.loads(message)["type"])

asyncio.run(main())
The event protocol is the provider’s own realtime protocol; the gateway doesn’t change it. For OpenAI, the gateway adds the OpenAI-Beta: realtime=v1 header to the upstream handshake. Sec-WebSocket-Protocol from your client is forwarded.

​
Common configurations

Use team model access and policies. Access policies, model restrictions, request rate limits and budgets are checked once, when the session opens. Per-request token ceilings don’t apply, because the gateway doesn’t read the session’s content.
Browsers can’t set an Authorization header on a WebSocket, and the gateway reads no credential from the query string. Open the session from your backend and relay audio to the browser.

​
What the gateway does and doesn’t do

AppliesDoesn’t apply
Authentication and team model accessGuardrails, firewall and PII redaction (frames aren’t inspected)
Access policies, model restrictions, rate limits, budgets (at connect)Caching
One audit record per session, with status and durationToken counting — sessions are metered with zero tokens
Request count and duration metricsFailover: only the model’s first upstream is used
Because frames pass through unmodified, nothing your organization configures for content — guardrails, PII redaction, the prompt-injection firewall — protects realtime sessions. The audit record stores [websocket] in place of request and response bodies.
Realtime models must be defined in gateway.yaml; console-registered workspace providers don’t serve /v1/realtime, so such a model returns 404 model_not_found. A non-openai/azure provider returns 400 unsupported_provider, and a request without Upgrade: websocket returns 400 websocket_upgrade_required. If the upstream can’t be reached, the client gets 502 upstream_error.

​
Next steps

Endpoints

Every path the gateway serves.

Policies

Rate limits and budgets checked at connect.

Providers

Configure OpenAI and Azure upstreams.

Making Requests

Authentication and errors.