Voice agents that run on the BEAM.

AudioFlow is an Elixir runtime for voice, image, and video agents. Each conversation is an OTP process. Speech recognition, language, and speech models run as shared services, on the same node or across a cluster.

One voice turn in the chat exampleApple M5, Qwen3-ASR 0.6B and Qwen3 0.6B, after the first turn

A process for each conversation. One copy of each model.

Sessions hold the turn, the history, and the output queue. Model weights load once into Nx.Servingprocesses that every session shares. Add sessions without adding GPU memory, and move a model to another node without changing a session.

How shared models work
<0.412.0><0.418.0><0.425.0><0.431.0><0.440.0><0.447.0>WhisperSpeech recognitionQwen3Reply textKokoroSpeech synthesis

Mix native and remote models in one session.

Each stage is an adapter. Point speech recognition at a local Whisper serving, the reply at a llama.cpp server, and speech at Kokoro. Swap any of them later without changing the session.

The owner process receives every transcript, text chunk, and audio chunk as a message. One call stops the reply and empties the speaker queue.

Read about sessions
{:ok, session} = AudioFlow.start_session(
  stt: {AudioFlow.NxAdapter, serving: MyApp.Whisper, ...},
  llm: {AudioFlow.Adapters.ReqLLM, model: "anthropic:claude-sonnet-4-5"},
  tts: {AudioFlow.NxAdapter, serving: MyApp.Kokoro, ...},
  output: {AudioFlow.Playback, playback},
  history: [max_turns: 8]
)

{:ok, turn} = AudioFlow.submit(session, %AudioFlow.Audio{data: pcm, sample_rate: 16_000})

receive do
  {AudioFlow, ^session, ^turn, event} -> handle(event)
end

# The user speaks again: stop the reply and flush the speaker.
AudioFlow.cancel(session)

What you get

Measured against Pipecat.

Both frameworks ran the same Silero, Whisper tiny, Qwen3-0.6B, and Kokoro services on one CPU, over 128 measured turns. Every turn completed, and transcripts and replies matched.

This compares orchestration over one inference engine. Pipecat called the models through local pipes, so its result includes that transfer cost.

See the method and full results
Median time to first output, in seconds
Simultaneous turnsAudioFlowPipecat
11.501.70
21.351.78
42.632.59
85.345.19

Run the demo in a minute.

The core needs no model downloads. Add AUDIOFLOW_ML=1 when you want real models.

$ git clone https://github.com/flowcopilot/audioflow.git
$ cd audioflow && mix deps.get
$ mix run examples/demo.exs