Adversarially Fine-tuned Conformers for SEF-MK Voice Anonymisation

Checkpoints and inference code for SEF-MK voice anonymisation, with Conformers fine-tuned against a speaker (and gender) adversary through gradient reversal. Speech in (wav/flac/ogg/mp3, any sample rate), anonymised 16 kHz speech out.

CC BY-NC-SA 4.0. Non-commercial use only. Derived from SEF-MK.

Quick start

hf download albertosaponaro/adv-ft-conformers-4-sef-mk-vas --local-dir adv-sef-mk
cd adv-sef-mk
pip install -r requirements.txt

python anonymize.py -i recording.flac -o anon/        # a file, a directory, or a .scp list
python anonymize.py -i my_audio/ -o anon/ --variant libri-20-sep_speaker
python anonymize.py --list-variants
from adv_sef_mk import Anonymizer

anon = Anonymizer.from_pretrained("albertosaponaro/adv-ft-conformers-4-sef-mk-vas")
anon.anonymize_file("recording.flac", "anonymised.wav")

Runs on GPU if available, otherwise CPU (about 15 s for 20 s of speech, including model loading). WavLM-Large (Microsoft, MIT) is included unchanged.

Samples

Three utterances from the LibriSpeech dev-clean set (CC BY 4.0).

Audio 1 Audio 2 Audio 3
Original
libri-all_speaker
libri-all_speaker-gender
libri-20-sep_speaker
libri-20-sep_speaker-gender

Variants and results

Each Conformer is paired with the k-means bank it was trained on (see variants.json). libri-all uses one codebook; libri-20-sep uses 58 (one per 20 speakers), drawn at random per utterance.

VoicePrivacy 2024 protocol, weak semi-informed attacker, EER averaged over five runs; dev/test averages.

Variant EER ↑ WER ↓ UAR ↑
Original speech 5.16 1.82 70.07
SEF-SK, no adversary 38.04 3.34 56.41
SEF-MK, no adversary (reproduced) 37.93 3.26 55.46
libri-all_speaker (default) 43.95 3.62 56.21
libri-all_speaker-gender 43.98 3.96 57.22
libri-20-sep_speaker 43.49 3.65 52.22
libri-20-sep_speaker-gender 43.19 3.57 54.98

EER: attacker's speaker verification on LibriSpeech (50 % = chance). WER: LibriSpeech ASR. UAR: IEMOCAP emotion recognition.

Limitations

Trained on read English speech (LibriSpeech). Anonymisation lowers speaker linkability under the attack above; it is not a guarantee of de-identification and does not remove spoken content.

The k-means codebooks are scikit-learn pickles, so the Hub's pickle scanner flags them. The Conformer checkpoints load with weights_only=True.

Licence and citation

CC BY-NC-SA 4.0, inherited from SEF-MK. WavLM, HiFi-GAN and the k-means helpers keep their MIT licences; see THIRD_PARTY_NOTICES.md.

@misc{saponaro2026advsefmk,
  title  = {Adversarial Finetuned Conformers for
            Speaker-Embedding-Free Multi-K-Means Voice Anonymization System},
  author = {Saponaro, Alberto},
  year   = {2026},
  url    = {https://huggingface.co/albertosaponaro/adv-ft-conformers-4-sef-mk-vas},
}

@inproceedings{tang2025sefmk,
  title     = {{SEF-MK}: Speaker-Embedding-Free Voice Anonymization
               through Multi-k-means Quantization},
  author    = {Tang, Beilong and Miao, Xiaoxiao and Wang, Xin and Li, Ming},
  booktitle = {2025 IEEE Automatic Speech Recognition and Understanding
               Workshop (ASRU)},
  year      = {2025},
  eprint    = {2508.07086},
  archivePrefix = {arXiv},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Dataset used to train albertosaponaro/adv-ft-conformers-4-sef-mk-vas

Paper for albertosaponaro/adv-ft-conformers-4-sef-mk-vas