Voice agents that run on the BEAM.
AudioFlow is an Elixir runtime for voice, image, and video agents. Each conversation is an OTP process. Speech recognition, language, and speech models run as shared services, on the same node or across a cluster.
Mix native and remote models in one session.
Each stage is an adapter. Point speech recognition at a local Whisper serving, the reply at a llama.cpp server, and speech at Kokoro. Swap any of them later without changing the session.
The owner process receives every transcript, text chunk, and audio chunk as a message. One call stops the reply and empties the speaker queue.
Read about sessions{:ok, session} = AudioFlow.start_session(
stt: {AudioFlow.NxAdapter, serving: MyApp.Whisper, ...},
llm: {AudioFlow.Adapters.ReqLLM, model: "anthropic:claude-sonnet-4-5"},
tts: {AudioFlow.NxAdapter, serving: MyApp.Kokoro, ...},
output: {AudioFlow.Playback, playback},
history: [max_turns: 8]
)
{:ok, turn} = AudioFlow.submit(session, %AudioFlow.Audio{data: pcm, sample_rate: 16_000})
receive do
{AudioFlow, ^session, ^turn, event} -> handle(event)
end
# The user speaks again: stop the reply and flush the speaker.
AudioFlow.cancel(session)What you get
A process per conversation
Each session is supervised. One crash ends one conversation, not the server.
Interrupt at any time
Cancel kills the turn task and flushes queued audio. Silero voice detection does it for you on barge-in.
Native inference
Whisper, Qwen3, Kokoro, VITS, CLIP, and BLIP run through Bumblebee, EXLA, and Transmuter.
Any language model server
Stream replies from llama.cpp or any ReqLLM provider, with tool calls and strict byte limits.
Routes across nodes
Send requests to bounded workers on other Erlang nodes. Change routes while sessions stay open.
Images and video
Describe RGB frames from MP4 files or cameras, and talk about them in later voice turns.
Memory with limits
History keeps complete turn pairs, capped by count and by bytes.
Back pressure everywhere
Every output chunk waits for an acknowledgement. Queues have fixed sizes.
Measured against Pipecat.
Both frameworks ran the same Silero, Whisper tiny, Qwen3-0.6B, and Kokoro services on one CPU, over 128 measured turns. Every turn completed, and transcripts and replies matched.
This compares orchestration over one inference engine. Pipecat called the models through local pipes, so its result includes that transfer cost.
See the method and full results| Simultaneous turns | AudioFlow | Pipecat |
|---|---|---|
| 1 | 1.50 | 1.70 |
| 2 | 1.35 | 1.78 |
| 4 | 2.63 | 2.59 |
| 8 | 5.34 | 5.19 |
Run the demo in a minute.
The core needs no model downloads. Add AUDIOFLOW_ML=1 when you want real models.
$ git clone https://github.com/flowcopilot/audioflow.git
$ cd audioflow && mix deps.get
$ mix run examples/demo.exs