Chat example
examples/chat is a local LiveView chat over one AudioFlow session per browser
tab. Type a message, or record one with the microphone. While you record, the
user message shows a live transcript. The reply text streams into the
conversation, and the reply audio plays as it arrives.
The app also exposes server-configured speech providers, languages, redacted recordings, playback progress, and provisional pause/resume. Its Linux demo needs no model download.
Its always-on modes use AudioFlow.Duplex with OpenAI Realtime or a local Moshi
server. The microphone stays open from Start to Stop, and the model audio plays at
the same time. OpenAI detects each utterance, replies by voice, and stops the reply
when you speak. See Remote speech providers
for the duplex adapters.
The models run on the Apple Silicon GPU through MLX. This example needs an Apple Silicon Mac with macOS 26.2 or later.
The repository mise.toml pins Erlang, Elixir, Node.js, and uv. The models
download through Elixir’s hf_hub and need no Python. Kokoro also needs espeak-ng, which mise does not
provide.
mise installbrew install espeak-ngcd examples/chatmix setupmix phx.serverOpen http://localhost:4000. Set PORT to use another port, and
LOG_LEVEL=debug to log each LiveView event.
Models
Section titled “Models”Select the models in the page. The first selection of a model downloads it. A switch applies to the next turn and keeps the conversation history.
| Stage | Models | Runtime |
|---|---|---|
| Speech recognition | Whisper tiny, small, large v3 turbo | Bumblebee on the EMLX GPU backend |
| Speech recognition | Qwen3-ASR 0.6B and 1.7B | a vLLM server at CHAT_ASR_BASE_URL, through ReqLLM |
| Reply | Qwen3 0.6B, 1.7B, 4B, and 8B, 4-bit | Native EMLXAxon Qwen3 generation |
| Speech | Kokoro 82M, four voices | Transmuter on the EMLX GPU backend |
The largest combination loads about 8.2 GB of weights. The defaults are Whisper
small, Qwen3 1.7B, and the af_heart voice. Set CHAT_STT, CHAT_LLM, or
CHAT_VOICE to change the startup selection.
Latency
Section titled “Latency”Measured on an Apple M5 with 24 GB of memory, after the first turn:
| Models | First text | First audio |
|---|---|---|
| Whisper small, Qwen3 1.7B | 0.9 s | 1.5 s |
The first turn after a model switch compiles the new model. It took up to 6 s to the first audio.
Bumblebee has no Qwen3-ASR model, so Qwen3-ASR needs a separate vLLM server,
for example vllm serve Qwen/Qwen3-ASR-1.7B. Set CHAT_ASR_BASE_URL when it
does not listen on http://localhost:8000/v1.
Model cache
Section titled “Model cache”Reply model checkpoints use the hf_hub cache, by default
~/.cache/huggingface/hub/models--OWNER--REPO/snapshots/REVISION. The example
pins each revision in Chat.Catalog. To use another cache directory, set
config :hf_hub, :cache_dir, "/absolute/path". Downloads from older versions,
under audioflow_chat in the user cache, are not reused.
Each selection checks the repository file list, so it needs access to the
Hugging Face Hub even when the files are cached. Complete files are reused, and
a failed download can be retried. After a crash, stale lock files can remain in
the cache’s locks/ directory. Remove them only when no download is running.
The browser records mono audio and converts it to 16 kHz s16le PCM. It sends new audio every 0.5 seconds, and the current model transcribes the recording so far. When you stop recording, the complete audio is one session turn. A recording stops after 30 seconds.
A new recording, a typed message, or the Stop button cancels the current reply
and stops playback. Microphone access requires localhost or HTTPS.