Voice, dictation, and speech

Chat supports dictation, hands-free voice calls, and text-to-speech playback. These controls depend on browser media support, microphone permission, a secure browser context, and compatible speech models.

Prepare speech models and controls

  1. Install or configure speech-to-text and text-to-speech models. The bundled Whisper and Melo TTS plugins provide local paths.
  2. In the SoAI WebUI, open Chat, open or start a conversation, then select Chat configuration in its header.
  3. Open Voice and choose automatic or specific STT and TTS models.
  4. Open Chat configuration > Appearance > Input actions, enable Voice and Voice call, then select Save.
  5. Open Chat through HTTPS or localhost and grant microphone permission when the browser asks.

Voice Call also requires a selected conversation model. A model used for STT or TTS does not replace that conversation model.

Set up speech-to-text

Media attachments, Knowledge ingestion, read_audio, and audio requested by read_video use a local speech-to-text plugin backend and model. Any plugin that publishes local audio transcription support can provide this capability. SoAI bundles the Whisper plugin. Microphone dictation and Voice Call can also use compatible remote providers.

These installation steps require Plugin read access (PLUGIN_READ), Plugin administration (PLUGIN_ADMIN), Model read access (MODEL_READ), and Model administration (MODEL_ADMIN).

  1. Open Plugins, select a speech-to-text plugin, and choose Install backend. Choose a compatible CPU or accelerator variant and start installation. The download can be several GB.
  2. Open Models > Add model > Download model, select the same plugin, and download a model. Smaller models are faster; larger models generally improve accuracy.

With Configuration changes (CONFIG_PATCH), choose the model in Settings > General > Speech-to-text (TOOLS.MCP.READ_VIDEO.WHISPER_MODEL). Fresh installations use Auto, selecting an eligible local model in model-ID order; existing installations retain their selection. A saved Whisper size keeps selecting that checkpoint. read_audio uses this setting when its model argument is omitted or null.

If no eligible model is available, select Add model to download one. Without MODEL_ADMIN, the control shows Not available. Searching for speech also opens the setting. Your saved selection is kept while its model is unavailable.

Download the model before enabling offline mode (SYSTEM.RUNTIME.STAY_OFFLINE). Offline mode blocks downloads while installed local models remain usable. Ask an administrator if a required permission is unavailable.

Files processed without usable speech setup retain their available metadata and visual extraction and show Speech not transcribed. Installing a backend or model does not change those earlier results. Add the file again to process it with the installed model. For a source in Files, use the Knowledge tab’s Import from Files action again. Reindex rebuilds embeddings from existing text; it does not transcribe earlier audio.

Use speech in Chat

Select Start voice recording, speak, then select Stop recording. SoAI transcribes the audio with the selected or automatic STT model and inserts the text into the composer. Review it before sending. Starting dictation ends an active voice call.

For push-to-talk, hold Start voice recording, speak, then release. SoAI transcribes the recording, adds it to any text already in the composer, and sends the message. Slide the pointer left before releasing, or press Esc, to cancel. If a reply is still running, the message joins the queue.

If you edit the draft or change the selected model during recording or transcription, the transcript stays in the composer for review instead of being sent. Switching conversations cancels the recording. Click the recording button without holding it to use ordinary dictation and review the text before sending.

Run a voice call

Select Start voice call. The call listens until you stop speaking, transcribes the utterance, sends it to the conversation model, and reads the reply through the selected TTS model. If you speak while the assistant is answering, SoAI stops the current speech and response so the new utterance can take over. Use Pause voice call, Resume voice call, or End voice call. Switching conversations ends the call.

Read one assistant message aloud

Hover or focus an assistant message and select Speak. It uses the same TTS choice configured for voice calls. The action appears only when speech playback applies to that message and the required model is available.

When a control is disabled

  • Use HTTPS or localhost; an insecure origin may not receive microphone access.
  • Allow the microphone and confirm that the browser supports media recording.
  • Select a conversation model before starting a call.
  • Check the configured STT or TTS model and its plugin state.
  • Enable the matching action under Chat configuration > Appearance > Input actions if the button is absent.

Related documentation