Adversarially Fine-tuned Conformers for SEF-MK Voice Anonymisation
Checkpoints and inference code for SEF-MK voice anonymisation, with Conformers fine-tuned against a speaker (and gender) adversary through gradient reversal. Speech in (wav/flac/ogg/mp3, any sample rate), anonymised 16 kHz speech out.
CC BY-NC-SA 4.0. Non-commercial use only. Derived from SEF-MK.
Quick start
hf download albertosaponaro/adv-ft-conformers-4-sef-mk-vas --local-dir adv-sef-mk
cd adv-sef-mk
pip install -r requirements.txt
python anonymize.py -i recording.flac -o anon/ # a file, a directory, or a .scp list
python anonymize.py -i my_audio/ -o anon/ --variant libri-20-sep_speaker
python anonymize.py --list-variants
from adv_sef_mk import Anonymizer
anon = Anonymizer.from_pretrained("albertosaponaro/adv-ft-conformers-4-sef-mk-vas")
anon.anonymize_file("recording.flac", "anonymised.wav")
Runs on GPU if available, otherwise CPU (about 15 s for 20 s of speech, including model loading). WavLM-Large (Microsoft, MIT) is included unchanged.
Samples
Three utterances from the LibriSpeech dev-clean set (CC BY 4.0).
| Audio 1 | Audio 2 | Audio 3 | |
|---|---|---|---|
| Original | |||
libri-all_speaker |
|||
libri-all_speaker-gender |
|||
libri-20-sep_speaker |
|||
libri-20-sep_speaker-gender |
Variants and results
Each Conformer is paired with the k-means bank it was trained on (see
variants.json). libri-all uses one codebook; libri-20-sep
uses 58 (one per 20 speakers), drawn at random per utterance.
VoicePrivacy 2024 protocol, weak semi-informed attacker, EER averaged over five runs; dev/test averages.
| Variant | EER β | WER β | UAR β |
|---|---|---|---|
| Original speech | 5.16 | 1.82 | 70.07 |
| SEF-SK, no adversary | 38.04 | 3.34 | 56.41 |
| SEF-MK, no adversary (reproduced) | 37.93 | 3.26 | 55.46 |
libri-all_speaker (default) |
43.95 | 3.62 | 56.21 |
libri-all_speaker-gender |
43.98 | 3.96 | 57.22 |
libri-20-sep_speaker |
43.49 | 3.65 | 52.22 |
libri-20-sep_speaker-gender |
43.19 | 3.57 | 54.98 |
EER: attacker's speaker verification on LibriSpeech (50 % = chance). WER: LibriSpeech ASR. UAR: IEMOCAP emotion recognition.
Limitations
Trained on read English speech (LibriSpeech). Anonymisation lowers speaker linkability under the attack above; it is not a guarantee of de-identification and does not remove spoken content.
The k-means codebooks are scikit-learn pickles, so the Hub's pickle scanner
flags them. The Conformer checkpoints load with weights_only=True.
Licence and citation
CC BY-NC-SA 4.0, inherited from SEF-MK. WavLM, HiFi-GAN and the k-means
helpers keep their MIT licences; see THIRD_PARTY_NOTICES.md.
@misc{saponaro2026advsefmk,
title = {Adversarial Finetuned Conformers for
Speaker-Embedding-Free Multi-K-Means Voice Anonymization System},
author = {Saponaro, Alberto},
year = {2026},
url = {https://huggingface.co/albertosaponaro/adv-ft-conformers-4-sef-mk-vas},
}
@inproceedings{tang2025sefmk,
title = {{SEF-MK}: Speaker-Embedding-Free Voice Anonymization
through Multi-k-means Quantization},
author = {Tang, Beilong and Miao, Xiaoxiao and Wang, Xin and Li, Ming},
booktitle = {2025 IEEE Automatic Speech Recognition and Understanding
Workshop (ASRU)},
year = {2025},
eprint = {2508.07086},
archivePrefix = {arXiv},
}