Skip to content

Chat example

examples/chat is a local LiveView chat over one AudioFlow session per browser tab. Type a message, or record one with the microphone. While you record, the user message shows a live transcript. The reply text streams into the conversation, and the reply audio plays as it arrives.

The app also exposes server-configured speech providers, languages, redacted recordings, playback progress, and provisional pause/resume. Its Linux demo needs no model download.

Its always-on modes use AudioFlow.Duplex with OpenAI Realtime or a local Moshi server. The microphone stays open from Start to Stop, and the model audio plays at the same time. OpenAI detects each utterance, replies by voice, and stops the reply when you speak. See Remote speech providers for the duplex adapters.

The models run on the Apple Silicon GPU through MLX. This example needs an Apple Silicon Mac with macOS 26.2 or later.

The repository mise.toml pins Erlang, Elixir, Node.js, and uv. The models download through Elixir’s hf_hub and need no Python. Kokoro also needs espeak-ng, which mise does not provide.

Terminal window
mise install
brew install espeak-ng
cd examples/chat
mix setup
mix phx.server

Open http://localhost:4000. Set PORT to use another port, and LOG_LEVEL=debug to log each LiveView event.

Select the models in the page. The first selection of a model downloads it. A switch applies to the next turn and keeps the conversation history.

Stage Models Runtime
Speech recognition Whisper tiny, small, large v3 turbo Bumblebee on the EMLX GPU backend
Speech recognition Qwen3-ASR 0.6B and 1.7B a vLLM server at CHAT_ASR_BASE_URL, through ReqLLM
Reply Qwen3 0.6B, 1.7B, 4B, and 8B, 4-bit Native EMLXAxon Qwen3 generation
Speech Kokoro 82M, four voices Transmuter on the EMLX GPU backend

The largest combination loads about 8.2 GB of weights. The defaults are Whisper small, Qwen3 1.7B, and the af_heart voice. Set CHAT_STT, CHAT_LLM, or CHAT_VOICE to change the startup selection.

Measured on an Apple M5 with 24 GB of memory, after the first turn:

Models First text First audio
Whisper small, Qwen3 1.7B 0.9 s 1.5 s

The first turn after a model switch compiles the new model. It took up to 6 s to the first audio.

Bumblebee has no Qwen3-ASR model, so Qwen3-ASR needs a separate vLLM server, for example vllm serve Qwen/Qwen3-ASR-1.7B. Set CHAT_ASR_BASE_URL when it does not listen on http://localhost:8000/v1.

Reply model checkpoints use the hf_hub cache, by default ~/.cache/huggingface/hub/models--OWNER--REPO/snapshots/REVISION. The example pins each revision in Chat.Catalog. To use another cache directory, set config :hf_hub, :cache_dir, "/absolute/path". Downloads from older versions, under audioflow_chat in the user cache, are not reused.

Each selection checks the repository file list, so it needs access to the Hugging Face Hub even when the files are cached. Complete files are reused, and a failed download can be retried. After a crash, stale lock files can remain in the cache’s locks/ directory. Remove them only when no download is running.

The browser records mono audio and converts it to 16 kHz s16le PCM. It sends new audio every 0.5 seconds, and the current model transcribes the recording so far. When you stop recording, the complete audio is one session turn. A recording stops after 30 seconds.

A new recording, a typed message, or the Stop button cancels the current reply and stops playback. Microphone access requires localhost or HTTPS.