Call Denoiser V4

AI disclosure: This project was built with AI assistance (Claude, by Anthropic) for writing code and documentation. All creative and technical decisions (model design, experiments, evaluation) are mine, and I reviewed everything before publishing.

🎧 Live demo Β· πŸ“‹ Part of the Real-Time Audio Denoisers project board, next to the DAW vocal denoisers (v1, v1c, v1b, v1d)

A causal, real-time speech denoiser for calls and meetings: 16 kHz, 10 ms algorithmic latency, 988k parameters, 0.15 ms of CPU per 5 ms block. The fourth version of my noise suppressor and the first to beat the noisy input on every metric.

Code, earlier versions and full write-up: GitHub: Audio_ML_Noise_Reduction

Trained on: the VoiceBank-DEMAND training set (CC BY 4.0) plus DNS Challenge clean speech mixed with DNS noise. Details below in Training data.

Results

VoiceBank-DEMAND test set (824 files, 16 kHz), evaluated once with the best dev checkpoint.

Metric Noisy input v2 v3 v4
PESQ (WB) 1.97 1.65 1.70 2.58
STOI 0.921 0.869 0.874 0.930
SI-SDR (dB) 8.45 13.36 13.17 17.21
Algorithmic latency 200 ms 200 ms 10 ms

Streaming vs offline parity: 108.3 dB. CPU per 5 ms block (ONNX Runtime): 0.15 ms (real-time factor 0.03).

Fair-comparison note: v1-v3 never saw VoiceBank-DEMAND; v4 trains on its training split (different speakers from the test set), which is the standard protocol for this benchmark and part of the jump.

Files

File What it is
call_denoiser_v4.onnx One-frame streaming model: magnitude [1, 1, 257] + GRU state [2, 1, 256] in, mask + new state out
call_denoiser_v4.pth PyTorch checkpoint (EMA weights, step 26,000)
model.py Model definition and asymmetric-window STFT
stream.py Real-time block engine (80-sample blocks in and out)

Usage

# pip install onnxruntime numpy
from stream import StreamEngine          # expects call_denoiser_v4.onnx next to it
eng = StreamEngine()
y_block = eng.process(x_block)           # float32[80] at 16 kHz -> 80 denoised samples, 10 ms later

from stream import process_file          # offline: whole mono 16 kHz array in, denoised array out
y = process_file(x)

Model

Asymmetric analysis/synthesis windows (Wang et al. 2021): 32 ms analysis window, 10 ms synthesis window, 5 ms hop. Per frame: compressed magnitude -> Linear -> LayerNorm -> 2-layer GRU (256) -> sigmoid magnitude mask; noisy phase kept (v3 showed a learned phase hurt quality).

Training: 26,208 steps (batch 32, 4 s segments) on an RTX 2060. Compressed spectral loss (Braun & Tashev) + multi-resolution STFT; AdamW, warmup + cosine, EMA 0.999.

Training data

Source Amount Use License
VoiceBank-DEMAND training set (Valentini-Botinhao et al.): paired clean/noisy speech, 28 speakers 11,572 utterances, about 9.4 h 60% of training examples; speakers p226 and p287 held out for dev CC BY 4.0
DNS Challenge fullband clean speech (subset) 13,733 clips, about 9.6 h 40% of examples, mixed on the fly with DNS noise at SNR -5 to 20 dB; 5% of files held out for dev Per source, see the DNS repo
DNS Challenge fullband noise (subset) 8,000 clips, about 20.5 h noise for the mixtures above Per source, see the DNS repo
VoiceBank-DEMAND test set (824 utterances, 2 unseen speakers) final evaluation only, never trained on or used to pick checkpoints CC BY 4.0

Everything resampled to 16 kHz mono (from 48 kHz).

Intended use and limits

  • Background noise suppression for voice calls, meetings and voice recordings, in real time on CPU.
  • Mono, 16 kHz. Resample other rates first.
  • Removes noise, not reverb. Very low SNRs (below -5 dB) and competing speakers are outside the training range.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using Stavion-Colquitt/Call_Denoiser_V4 1