Membrane input
AUDIOFLOW_MEDIA=1 mix deps.getAUDIOFLOW_MEDIA=1 mix compile --warnings-as-errorsPCM sink with voice detection
Section titled “PCM sink with voice detection”AudioFlow.PCMSink accepts mono :s16le raw audio. With its vad option, it
submits utterances after endpoint detection:
%AudioFlow.PCMSink{ session: session, vad: {AudioFlow.Silero, serving: AudioFlow.VAD}, endpoint_options: [min_speech_ms: 96, min_silence_ms: 320]}Start AudioFlow.VAD once with AudioFlow.Silero.serving/2 and
AudioFlow.Models.child_spec/3. The Silero source can be a local ONNX file or
{:hf, "onnx-community/silero-vad", "onnx/model.onnx"}. Each sink keeps its
own recurrent state; only the weights are shared. Input must be 16 kHz. Pin the
model with load_options: [revision: "e71cae966052b992a7eca6b17738916ce0eca4ec"].
Put a resampler before the sink for other rates.
Barge-in
Section titled “Barge-in”After speech confirmation, the sink cancels the active reply and sends
{:audio_flow_speech_started, session} to its parent. A configured output
transport is flushed by the session. Otherwise, the parent must flush playback.
After endpoint detection, the sink sends {:audio_flow_input, result}. The parent
must handle busy input. At end of stream, the sink flushes the final utterance.
Without vad, each input buffer must be one complete utterance.
Without Membrane
Section titled “Without Membrane”AudioFlow.VoiceInput supplies the same frame processing without Membrane. It
keeps pre-roll, confirms speech, uses separate start and stop thresholds, and
limits utterance length. It pads a partial final frame with silence. Endpoint
options are documented in the module.
AUDIOFLOW_ML=1 AUDIOFLOW_PCM=recording.s16le mix run examples/vad.exsComplete input chain
Section titled “Complete input chain”AudioFlow.Input.attach/3 adds FFmpeg resampling and the session sink to a
Membrane source chain. It converts mono or stereo input to mono s16le at the
selected rate (16 kHz by default). AudioFlow.InputPipeline supplies this
chain as a complete pipeline. The source can be AudioFlow.PCMFileSource or
Membrane.PortAudio.Source. The file source reads bounded chunks and preserves
sample timestamps. It rejects incomplete PCM frames. Set the source format to match
the actual file or device.
AudioFlow.Input.attach/3 requires a voice detector tuple.