audiosronnx-voiceisolator

ONNX graphs of Scicom's voice isolator (Zolkepli, The Voice on the Line) for the voiceisolator engine in audiosronnx.

The model keeps one speaker and removes other voices and noise: target-speaker extraction, 16 kHz in and out, causal and streaming with 10 ms of delay. A speaker vector from an enrollment clip, or from the loudest 3 s of the input, tells it whose voice to keep.

Folder Model Step graph Source
/ CPU release, 22 M 85 MB Scicom-intl/voice-isolator-cpu
int8/ CPU release, 8-bit 25 MB same
agent/ CPU agent model (presence gate) 86 MB same
agent/int8/ CPU agent model, 8-bit 25 MB same
large/ GPU release, 555 M 2.2 GB (step.onnx.data) Scicom-intl/voice-isolator
large/agent/ GPU agent model (presence gate) 2.2 GB (step.onnx.data) same

Each folder holds config.json, step.onnx (one streaming step) and enroll.onnx (the speaker encoder). The step graph takes the STFT of new frames and every piece of stream state, and returns the masked spectrum and the next state. The runtime keeps the STFT, the overlap-add and the attention rings in numpy.

The CPU folders are the upstream graphs, unchanged, at revision 1dfac59. The large/ folders are exported from upstream revision d41d6d8 with conversion/export_voiceisolator.py. Their step graph also carries WavLM-Large, made causal, and takes a fixed 50 frames (0.5 s) per step.

Parity

Graphs Reference Max abs err
CPU, all four folders upstream voice_isolator/onnx_stream.py, 22 s 0.0
large/ upstream PyTorch stream_step, 14 s, float32 7.5e-8
large/agent/, gate on and off upstream PyTorch stream_step, 14 s, float32 1.2e-7

The 14 s runs go past the point where the 10 s attention rings wrap.

Usage

from audiosronnx import load_denoise

iso = load_denoise("voiceisolator")                         # CPU release
iso = load_denoise("voiceisolator", model="agent")          # presence gate
iso = load_denoise("voiceisolator", quantized=True)         # 8-bit
iso = load_denoise("voiceisolator", model="large")          # 555 M
iso = load_denoise("voiceisolator", enrollment="me.wav")    # lock onto a voice

clean, rate = iso.denoise("call.wav")                       # rate == 16000

License

Upstream declares license: other and publishes no license text. The terms of the upstream repositories apply to these files.

Downloads last month
33
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including TigreGotico/audiosronnx-voiceisolator

Paper for TigreGotico/audiosronnx-voiceisolator