Production integration

A production client should know only its application conversation, gateway, and approved product profile. Provider credentials, system/developer prompts, model, voice, temperature, token ceilings, reasoning, tool permissions, storage keys, and retention policy belong on the server.

Trust boundary

Boundary	Required control
Browser / app → gateway	Authenticate the user; verify conversation ownership on every request
Upstream body	Never pass through client JSON; extract user content and rebuild the body server-side
Model and prompt	Lock model and system/developer prompts; reject privileged client roles
Generation and voice	Lock/allowlist voice, speed, temperature, max tokens, reasoning, and tools
Cross-site requests	Enforce exact Origin; use SameSite cookies and a CSRF token when cookies authorize requests
Abuse	Limit by user, conversation, profile, IP, concurrency, and provider-cost budget
Input	Cap HTTP body size and the SDK’s maxTurnMs; reject unsupported content types
Output	Set upstream and total request timeouts; abort work when the client disconnects

The runnable Audio LLM-only example contains no client model, prompt, voice, or generation controls. Its server rebuilds requests with createOpenRouterGateway(), validates Origin, and bounds request/history/text sizes. Connect the mandatory authorize hook to your user and conversation auth; the example is not an account service.

Standard client factory	Base URL	Server locks
createOpenRouterGatewayASR	/api/voice/asr	ASR model and language policy
createOpenRouterGatewayAudioLLM	/api/voice/audio-llm	model, system prompt, voice, temperature, tokens
createOpenRouterGatewayVoiceTurn	/api/voice/asr-llm-tts	ASR, LLM, and TTS policy; one SSE response returns input transcript, text, and MP3 segments

The composite voice factory implements AudioLLMProvider with transcribesInput: true. With createOtterVoiceSession, clients may omit caption asr; the gateway must still authorize, rate-limit, and meter the asr_llm_tts profile independently.

Credentials and headers

• Never place a long-lived key in browser code, app binaries, EXPO_PUBLIC_*, or an API response.
• Do not put a shared “gateway password” in client environment variables. Send the current user’s short-lived application session and validate conversation ownership server-side.
• Use a same-origin policy gateway for request/response APIs. If a client must connect directly to a streaming provider, issue only short-lived route/model/budget-scoped tokens.
• Validate custom request-header values before sending them. Browser headers must contain Web-compatible characters; encode arbitrary metadata in a JSON body instead.
• Redact authorization, cookies, signed URLs, provider payloads, and Base64 audio from traces.

Turn audio and private storage

Use the final audio events; do not wrap provider methods or correlate temporary variables:

session.on('user_audio_final', async ({ turnId, audio, format }) => {
  await uploadPrivateTurn({ turnId, role: 'user', bytes: audio, format });
});

session.on('assistant_audio', async ({ turnId, audio, mimeType }) => {
  if (audio) await uploadPrivateTurn({ turnId, role: 'assistant', bytes: audio, format: mimeType });
});

• Store recordings in a private bucket. Object keys must be unguessable and scoped to your tenant/conversation record.
• Play saved recordings through an authenticated endpoint or a short-lived signed URL. Never make the bucket public.
• Store the object key, checksum, retention deadline, and turn id in the same auditable workflow as the database turn.
• On conversation/account deletion, enqueue object deletion, retry it idempotently, and record completion. A database cascade does not delete bucket objects.
• Obtain consent and define retention before enabling recording. Treat transcripts and voice as sensitive personal data.

raw and cause on errors can contain upstream responses or user data. They are development diagnostics, not production log fields. Log code, stage, provider, httpStatus, retryable, fatal, safeMessage, and your own request id.

Retry policy

Retry only failures marked retryable; use exponential backoff with jitter and honor provider retry headers at the gateway. Do not retry authentication, quota, audio-decode, or request-validation failures without changing the request. Client audioLlmRetry, rolling ASR, after_audio, and backend selection can all increase calls/spend; server budgets and idempotency must not trust those client values.

const session = createOtterVoiceSession({
  // Standard clients make one request; the gateway enforces budget independently.
  audioLlmRetry: { maxAttempts: 1 },
  // ...
});

Core retries a turn only before any transcript/audio stream output is delivered. Once output starts, an automatic retry could duplicate speech, so the failure is reported instead. With continueSessionOnFailure, the error event has fatal: false and the session returns to listening; otherwise it enters the terminal error state.

Launch checklist

• Test 401/403, 402/quota, 429, 5xx, timeout, client abort, malformed SSE, decode failure, and playback failure separately.
• Export latency from user_audio_end to assistant_audio_start, plus error stage and HTTP status.
• Run bun run test:smoke:audio-llm in CI. It uses fixed WebM bytes and no microphone while exercising conversion, the gateway, SSE parsing, and final audio.
• Set privacy-safe alert samples. Never attach whole request/response bodies automatically.
• Document user deletion, retention, provider region, and incident-response ownership before launch.