Voice (preview)

Preview. Voice is available in ChatAI 1.1 as a preview. It is usable in production, but its behavior, settings and client APIs may change in minor releases, and it is not yet covered by the same stability guarantees as text chat.

Voice (preview) lets visitors speak to an assistant from the same widget they type in. Answers come from the same Purpose, Instructions and Knowledge as text chat, and the conversation continues seamlessly when the visitor switches between typing and speaking.

How it works

1. The visitor presses the microphone button. The widget asks ChatAI for a Voice session; ChatAI applies the same domain allowlist, rate limits, bot checks and widget signing as text chat, plus usage limits.
2. The browser connects to the realtime voice model (GPT-Live) over WebRTC with a short-lived session. Provider keys never reach the browser.
3. ChatAI supervises the call from the server over a separate control link and decides who answers each thing the visitor says (see below). The text answer is the canonical record.
4. When the call ends, ChatAI settles the usage and, if enabled, saves the transcript and the recording.

Who answers each turn

ChatAI's server, not the voice model, decides what the visitor hears:

1. ChatAI classifies every utterance. Only greetings, thanks, goodbyes and short acknowledgements ("okay", "got it", "mhm") count as small talk. Everything else goes to ChatAI's backend, including statements ("I'm hungry"), single words, feelings, "What do you mean?", requests to repeat, and "yes" in answer to a question.
2. Small talk: the voice model may answer it directly.
3. Everything else: ChatAI's text answer pipeline answers it (Scope Router, Purpose, Profile, history, retrieval and the output guard), using the assistant's hallucination mode as in text chat. If the voice model doesn't hand the turn to ChatAI within a fraction of a second, or starts answering on its own, ChatAI runs the pipeline anyway and tells the model to say only that answer.
4. Playback gate: the widget plays the voice model's audio, and shows its captions, only after ChatAI approves them. Anything the model says before ChatAI's answer is muted, kept out of the conversation history, and labelled Withheld (not heard) in Conversation Review. The gate closes again as soon as the visitor starts speaking, so interruptions work as before.

The voice model's instructions repeat these rules, but they are a backup: the server-side gate is what enforces them.

Voice runtime state lives in the web process, so Voice needs a long-lived Node.js server (the Docker image or next start) and a single web instance or sticky routing. Serverless hosting is not supported for Voice; text chat keeps working there. See Voice in production.

Set up Voice (instance)

Set the server-side key and restart:

VOICE_OPENAI_API_KEY=sk-...        # OpenAI project key with realtime access
# optional
VOICE_MAX_CONCURRENT_SESSIONS=10   # instance-wide cap
VOICE_IDLE_TIMEOUT_SECONDS=180

Without a key, the Voice switch has no effect and the widget never shows a microphone. Full list: Environment reference.

Turn Voice on (assistant owner)

In the assistant's Settings → Privacy:

If Store visitor conversations is off, Voice stays ephemeral: nothing is saved and nothing is recorded, regardless of the other settings.

Try it first from the assistant Playground.

Recordings and consent

Recording needs S3-compatible object storage (AWS S3, Cloudflare R2, MinIO) configured with the OBJECT_STORAGE_* variables. Without it, Record Voice audio is disabled and Voice works normally.

When recording is on:

• Before the microphone turns on, the widget shows a recording notice. The visitor must choose Agree and start for each call; Cancel returns to typing without starting Voice.
• While recording, the Voice bar shows Recording.
• Each call produces one stereo WebM/Opus file (visitor left, assistant right), about 0.36 MB per minute. Only compressed audio is written to the server's spool; raw audio never touches disk.
• Recordings are private. The bucket is never exposed to browsers; owners play recordings back in Conversations through an authenticated proxy, with the transcript timeline alongside.
• Recordings are deleted with their conversation, by the recording retention setting, and when the assistant is deleted.
• If saving a recording fails, the call itself is not affected.

Storage setup and a MinIO example: Environment reference.

Usage minutes and limits

Voice usage is measured in seconds per call and follows HOSTED_USAGE_ENFORCEMENT (see Billing and usage):

• shadow (default) and off record usage without refusing calls, apart from VOICE_MAX_CONCURRENT_SESSIONS.
• enforce reserves time from the account's Voice-minute allowance in 5-minute blocks while the visitor talks. New calls are refused when fewer than 30 seconds remain or when the plan's concurrent-call limit is reached. A call that reaches the end of the allowance is warned about 30 seconds before it ends.

Other limits:

• Calls end after 60 minutes (with a warning about 30 seconds before) and after the idle timeout.
• Starting Voice in a second tab ends the call in the first tab.
• Only one input runs at a time: typing is paused during a call, and Voice cannot start while a text reply is streaming.

When Voice is refused or ends early, visitors see Voice isn't available right now or Voice is unavailable for the rest of this conversation, but we can keep chatting here. and can keep typing. Visitors and the assistant are never told about plans, billing, usage or remaining minutes.

Hide Voice on a page

Voice appears automatically when the assistant offers it. To hide it on a specific page:

<script
  src="https://app.nightzeros.com/widget/chat.js"
  data-assistant-id="asst_your_public_id"
  data-voice="off"
  async
></script>

<ChatWidget assistantId="asst_your_public_id" apiUrl="https://app.nightzeros.com" voice={false} />

Voice needs a widget that supports the playback gate. The hosted chat.js always does once the browser has loaded the current copy; it is cached for at most 5 minutes after a deploy. npm users need @nightzeros/chatai-react or @nightzeros/chatai-widget 1.2.0 or later. ChatAI refuses Voice to older widgets, which show Voice needs an updated chat widget. Reload the page, or ask the site owner to update it. and keep text chat working.

The Playground uses the same playback gate as the widget: what you hear while testing is what visitors hear.

Browser requirements

• HTTPS (or localhost), microphone permission and WebRTC. When any is missing, the widget explains why and text chat keeps working.
• If your page sets a Permissions-Policy header, it must allow microphone for your origin. Pages inside an iframe need allow="microphone" on the iframe.
• Closing the panel, navigating away or removing the widget ends the call and releases the microphone.

Troubleshooting

Privacy notes

• The voice model receives the assistant's Purpose (owner text) and the conversation, not your Knowledge base; Knowledge reaches it only through ChatAI's answers.
• Transcripts and recordings follow the assistant's privacy settings and retention.
• Recording requires explicit visitor consent for every call.
• Provider keys, plan details and usage figures are never sent to the browser.

Known limitations

• The voice model's audio still reaches the browser; the widget mutes it until ChatAI approves. Modified client code could unmute withheld audio. Only relaying all audio through the server would remove that, and that is out of scope for the preview.
• After ChatAI approves an answer, the voice model speaks it in its own words and could add to it. The stored answer is always ChatAI's text, and spoken additions never become grounded history.

Related

• Widget and React
• Assistants: Scope and Profile
• Docker: Voice in production