MMS-1B adapter for Ngen (gnj), elicited speech

An adapter for facebook/mms-1b-all that transcribes Ngen (also known as sa̰wa̰la̰ŋ́, ngain; Southern Mande, Côte d'Ivoire) from 16 kHz audio. It was trained on 72 minutes of manually checked fieldwork recordings and is the model used to make first-pass transcriptions in Tatiana Korol's Ngen documentation database.

  • Output: lowercase Ngen in the working orthography, without tones and without punctuation (see Output format).
  • Test results: CER 0.099, WER 0.277. 41 % of test segments are transcribed with no errors at all.
  • Size: about 8 MB (adapter layers and the output layer). You also need the 1B-parameter base model.

How to use

import torch, soundfile as sf
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

REPO = "bomolopuu/mms-1b-ngen-elicit"

processor = Wav2Vec2Processor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(
    "facebook/mms-1b-all",
    vocab_size=len(processor.tokenizer),
    pad_token_id=processor.tokenizer.pad_token_id,
    ignore_mismatched_sizes=True,
)
model.init_adapter_layers()
state = load_file(hf_hub_download(REPO, "adapter.ngen.safetensors"))
model.load_state_dict(state, strict=False)
model.eval()

audio, sr = sf.read("clip.wav", dtype="float32")   # mono, 16 kHz
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(processor.batch_decode(logits.argmax(-1))[0])

Practical notes from our own use:

  • Use one phrase per clip (roughly 0.3–20 s). Longer audio should be cut with a voice activity detector first.
  • Pad the clip around the speech (we add ~400 ms before and ~200 ms after). Clips that start right at the first sound often lose a quiet initial syllable or particle.
  • Greedy CTC decoding; no language model is used.

Output format

The vocabulary has 29 characters plus a word separator:

a b c d e f g h i j k l m n o p r s t u v w y z  ŋ ɔ ɛ ɲ  ◌̰
  • Nasalization is written with a combining tilde below (U+0330) after the vowel (a̰ ḭ ṵ ɛ̰ ɔ̰), in decomposed form.
  • Tones are not transcribed. In our workflow tones are added afterwards by a linguist.
  • No capitals, punctuation or digits.

Training data

  • 72.0 minutes of elicited Ngen speech from fieldwork recordings: 4,312 segments, all with transcriptions checked by hand. Another 601 segments (~10 %) were used as a development set for early stopping.
  • Three adult speakers (anonymized here as S, Y and E) with very unequal amounts of data: S 45.1 min, Y 18.7 min, E 7.0 min.
  • Mostly elicitation (words and short sentences given in response to French prompts), plus a small amount of narrative speech.
  • The recordings and transcriptions are not released with this model.

Training procedure

  • Base model: facebook/mms-1b-all, frozen. Only the language adapter layers and the output (CTC) layer were trained.
  • The adapter was initialized from the MMS adapter for Yaure (yre), a related Southern Mande language. With 72 minutes of data this choice barely matters: initializing from Bambara or from scratch gives the same result within seed variation.
  • Learning rate 1e-3, warmup ratio 0.1, batch size 8 × 2 gradient accumulation, up to 40 epochs, time masking 0.05, fp16. Early stopping on development-set CER (patience 5). The best checkpoint was at epoch 38.
  • Out of five training seeds, this one (seed 2) was chosen by its development-set CER, not by its test score.
  • The training set was checked for overlap with the test set by audio file hash and time interval as well as by text. Every segment that shared audio or a sentence with the test set was removed from training.

Evaluation

The held-out test set has 625 segments (12 min) from the same three speakers. It was fixed before training and never used for model selection.

CER WER exact segments
all 0.099 0.277 41.3 %
speaker S (45 min in training) 0.065 — 66.0 %
speaker Y (19 min in training) 0.103 — 31.8 %
speaker E (7 min in training) 0.177 — 14.7 %

Errors closely track how much training data each speaker has. The main error types are:

  • word boundaries (about 23 % of all character edits);
  • vowel height (e/i, e/ɛ, o/u, o/ɔ);
  • lost nasalization: more than half of the nasalized vowels lose it.

When a linguist went through all 367 test segments with errors, more than half of the "errors" turned out not to be the model's fault. They came from inconsistent spelling in the reference transcriptions, wrong references or bad segment boundaries.

Limitations

  • Not tested on new speakers. All test speakers are also in the training data, so expect higher error rates for other voices, especially for children or different dialects.
  • Elicited speech. Fast narrative or conversational speech, overlapping speakers and noisy recordings give noticeably worse output.
  • No tones. The output can't be used as a tonal transcription.
  • French stimuli. French speech in the clip (for example the elicitation prompt) comes out as nonsense Ngen.
  • The model is meant to give a draft for a linguist to correct. It is not a finished transcription.

License

Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0).

You may use, share and adapt this model for any non-commercial purpose as long as you credit the author (see below) and indicate if you made changes. Commercial use is not allowed. The base model facebook/mms-1b-all is under the same license.

Citation

If you use this model, please cite:

@misc{korol2026ngenasr,
  author       = {Tatiana Korol},
  title        = {{MMS-1B} adapter for {Ngen} speech recognition},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/bomolopuu/mms-1b-ngen-elicit}}
}

and the MMS paper:

@article{pratap2023mms,
  title   = {Scaling Speech Technology to 1,000+ Languages},
  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and Tomasello, Paden and Babu, Arun and Kundu, Sayani and Elkahky, Ali and Ni, Zhaoheng and Vyas, Apoorv and Fazel-Zarandi, Maryam and Baevski, Alexei and Adi, Yossi and Zhang, Xiaohui and Hsu, Wei-Ning and Conneau, Alexis and Auli, Michael},
  journal = {arXiv preprint arXiv:2305.13516},
  year    = {2023}
}

Acknowledgements

The model exists thanks to the Ngen speakers who shared their language during fieldwork in Côte d'Ivoire. The recordings were transcribed and the model was trained at Humboldt-Universität zu Berlin.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bomolopuu/mms-1b-ngen-elicit

Finetuned
(449)
this model

Paper for bomolopuu/mms-1b-ngen-elicit

Evaluation results

  • cer on Ngen elicitation, held-out test (625 segments, 12 min)
    self-reported
    0.099
  • wer on Ngen elicitation, held-out test (625 segments, 12 min)
    self-reported
    0.277