Voxira Yorรนbรก Phase XII: Yorรนbรกโ€“English code-switching ASR

Phase XII = newly trained acoustic weights + a rebuilt 4-gram KenLM + a pyctcdecode bundle. The acoustic model is a 10-epoch fine-tune of facebook/wav2vec2-large-xlsr-53; the language model is a pruned 4-gram trained on a 141,476-line corpus balanced 40 / 30 / 30 (Yorรนbรก / English / code-switched). Unlike Phase XI, which kept the Phase X acoustic weights and changed only the decoder, Phase XII changes both. Decoding requires a blank-fusion step (see below); plain processor.batch_decode will not work.

Status: release candidate. Decoder comparison complete; independent held-out verification pending. The numbers below come from a small held-out set of read studio recordings (one corpus) on which decoder settings were also compared. They are not a generalisation estimate.

At a glance (all 49 valid held-out clips, 749 words): greedy with blank fusion 26.70% WER โ†’ KenLM at default settings 19.76% (26% fewer word errors). The shipped configuration scored 18.52% on the 44 clean takes (greedy there: 26.96%), but its word list and settings were tuned on those same clips, so the default-settings figure is the more honest one.

What changed from Phase X / XI

Phase X Phase XI Phase XII
Acoustic weights Phase 10 fine-tune Identical to Phase X Newly trained (checkpoint-2950, 10 epochs)
Output vocabulary 42 tokens, no Q / X / Z (per the Phase XI card) same acoustic model 46 entries in vocab.json (adds Q=42, X=43, Z=44, '=45) + <pad> at row 46 โ†’ 47 output rows
Blank handling standard processor decoding standard processor decoding blank probability split across rows 0 and 46; fusion required
LM text ~366k lines, ~85.6% Yorรนbรก / ~1.2% English 50k lines, entropy-balanced, templated openings capped, banking/conversational text up-weighted 4ร— 141,476 lines, 40 / 30 / 30, boilerplate removed, templated openings capped at 100 per prefix
KenLM 5-gram 4-gram c50k_prune, 4.1 MB, --prune 0 1 1 2 4-gram phaseXII_40_30_30_prune.bin, 10.83 MB, --prune 0 1 1 2
Unigram list large unigram list 47,017 words (lm_vocab) 50,259 words
alpha / beta 2.0 / 1.5 0.3 / 0.5 0.25 / 0.5 stored in attrs.json (0.30 / 0.50 also tested)

Evaluation: read this honestly

Held-out set. The last 50 unique prompts of the AR_metadata code-switching corpus (AR_396.wavโ€“AR_449.wav) were held out and removed from every LM training bucket. Of those 50 audio files, AR_418.wav is empty (0 bytes), leaving 49 valid clips; 5 of those are aborted or cut-off studio takes (AR_409.wav, AR_414.wav, AR_424.wav, AR_430.wav, AR_440.wav), leaving 44 clean takes. The 44 clean takes are a subset of the 49, so the two tables are not independent.

44 clean completed takes (664 reference words)

Decoder Errors / words WER 95% CI (naive) vs greedy Relative error reduction Selection status
Greedy CTC, blank-fused, no LM 179 / 664 26.96% 23.7โ€“30.5% baseline โ€” baseline
KenLM, 66,184-word list, alpha=0.30 beta=0.50 128 / 664 19.28% 16.5โ€“22.5% -7.68 pp 28.5% (51 fewer errors) Phase XI default settings; word list from the training corpus only
KenLM, 66,184-word list, alpha=0.25 beta=0.50 125 / 664 18.83% 16.0โ€“22.0% -8.13 pp 30.2% (54 fewer errors) settings chosen by comparing on this set
KenLM, 66,184-word list, alpha=0.35 beta=0.60 125 / 664 18.83% 16.0โ€“22.0% -8.13 pp 30.2% (54 fewer errors) settings chosen by comparing on this set
KenLM, expanded word list, alpha=0.30 beta=0.50 125 / 664 18.83% 16.0โ€“22.0% -8.13 pp 30.2% (54 fewer errors) word list extended after inspecting errors on this set
KenLM, expanded word list, alpha=0.25 beta=0.50 123 / 664 18.52% 15.8โ€“21.7% -8.43 pp 31.3% (56 fewer errors) shipped word list + settings; both tuned on this set
KenLM, expanded word list, alpha=0.35 beta=0.60 123 / 664 18.52% 15.8โ€“21.7% -8.43 pp 31.3% (56 fewer errors) word list and settings tuned on this set

All 49 valid clips, including aborted takes (749 reference words)

Decoder Errors / words WER 95% CI (naive) vs greedy Relative error reduction Selection status
Greedy CTC, blank-fused, no LM 200 / 749 26.70% 23.7โ€“30.0% baseline โ€” baseline
KenLM, 66,184-word list, alpha=0.30 beta=0.50 148 / 749 19.76% 17.1โ€“22.8% -6.94 pp 26.0% (52 fewer errors) Phase XI default settings; word list from the training corpus only
KenLM, 66,184-word list, alpha=0.25 beta=0.50 145 / 749 19.36% 16.7โ€“22.3% -7.34 pp 27.5% (55 fewer errors) settings chosen by comparing on this set
KenLM, 66,184-word list, alpha=0.35 beta=0.60 145 / 749 19.36% 16.7โ€“22.3% -7.34 pp 27.5% (55 fewer errors) settings chosen by comparing on this set

Why blank fusion is required (5-clip development probe, 118 words)

Configuration Errors / words WER Change
Greedy, row 46 masked, NOT fused into row 0 54 / 118 45.76% โ€”
Greedy, rows 0 + 46 fused (logaddexp) 39 / 118 33.05% -12.71 pp vs unfused (27.8% fewer errors)
Fused + KenLM (alpha=0.30, beta=0.50) 27 / 118 22.88% -10.17 pp vs fused greedy (30.8% fewer errors)

Frame-level argmax on one clip (Track-18.wav) showed row 2 (|, 195 frames) and row 46 (<pad>, 98 frames) dominating. Masking row 46 without fusing it into row 0 pushed blank frames onto row 0 or row 2, which injected literal PAD strings and stray spaces into words (LEPADCTURE NOWPADTES, EPADXAPADM). On the same probe, raising alpha to 0.60โ€“0.95 gave 26.27%โ€“33.9% WER because the LM began overriding correct diacritics and merging compound English nouns. The probe was used while debugging the decoder; it is not an independent estimate.

Example clips

Selection rule: the two clips where the LM helped most, the median clip, and the two where it helped least or hurt (sorted by WER change).

Clip Reference Greedy (fused) KenLM Greedy WER KenLM WER
AR_439.wav MO RA TRAVEL INSURANCE NรTORร Pร‰ รŒRรŒNร€Jร’ ร’ JแปŒ OHUN Tร A Lรˆ FOJรš Kร‰Rร‰ Sร MOO TRAVEL INSแนขURANCE NรTORร Pร‰ รŒRรŒNร€Jร’ OJรšHร™N TI ALรˆ FรšJรš K MO TRAVEL INSURANCE NรTORร Pร‰ รŒRรŒNร€Jร’ OJรš OHUN TI A Lรˆ FOJรš E 75.0% 37.5%
AR_428.wav WแปŒN Bร‰รˆRรˆ FรšN WRITTEN STATEMENT MI WแปŒN Bร‰รˆRรˆ FรšN WRITEN STATEMENT M WแปŒN Bร‰รˆRรˆ FรšN WRITTEN STATEMENT MI 33.3% 0.0%
AR_420.wav MO Nร LรTI LแปŒ Sร HEAD OFFICE LแปŒLA NรTORร Pร‰ WแปŒN TI BLOCK GBOGBO ONLINE TRANSACTIONS Tร MO Fแบธ แนขE MO Nร LรTI LแปŒ Sร HEAD OFFICE LแปŒLA NรTORร Pร‰ WแปŒN TI BLOCK GBOGBO ONLINE TRANSACTIONS Tร MO Fแบธ MO Nร LรTI LแปŒ Sร HEAD OFFICE LแปŒLA NรTORร Pร‰ WแปŒN TI BLOCK GBOGBO ONLINE TRANSACTIONS Tร MO Fแบธ แนขE 5.0% 0.0%
AR_413.wav GBOGBO ร€WแปŒN Tร A INVITE Sร WEDDING RECEPTION YแบธN GBแปŒDแปŒ MแปŒRA Kร WแปŒN Mร BAร€ MISS OUT Lร“Rร BUFFET แปŒSรN Nรร€ GBGBO ร€WแปŒN Tร A INVITE Sร WEDDING RECEPTION YแบธN GBแปŒGUN MแปŒRA Kร WแปŒN Mร BรŒ MISS OUT Lร“Rร BUFFET แปŒSรN GBOGBO ร€WแปŒN Tร A INVITE Sร WEDDING RECEPTION YแบธN GBOGUN MแปŒRA Kร WแปŒN Mร BA MISS OUT Lร“Rร BU FE แปŒSAN N 19.0% 28.6%
AR_446.wav Bร A Bร PรN HOUSEHOLD RESPONSIBILITIES รŒGBร‰SรAYร‰ WA ร RแปŒRร™N B A Bร PรN HOUSEHOLD RESPONSIBILITIES รŒGBร‰SรAYร‰ WA ร RRร™ Bร A Bร PรN HOUSEHOLD RESPONSIBILITIES รŒGBร‰Sร AYร‰ WA Rร™N 20.0% 40.0%

Caveats

  • Small and narrow. 749 words from one corpus of read studio recordings; speaker count is not recorded. Naive 95% intervals above are about ยฑ3 pp and are too narrow because errors cluster within clips.
  • Tuned on the evaluation clips. The alpha/beta comparison was run on these clips, and the shipped unigram list (50,259 words) includes 3,000 extra words (common English from the Brown corpus plus a hand-curated list). The curated words were added after inspecting errors on these clips (for example RABBIT after the AR_407.wav error). The rows marked "tuned" are therefore not clean held-out numbers. The cleanest row is the one with Phase XI's default settings and the corpus-only word list.
  • Domain match. The LM text includes the 376 training prompts from the same AR_metadata corpus, repeated 15ร— (none of the 50 held-out prompts). The held-out clips come from the same prompt set, which favours the LM compared with unseen speech.
  • Not comparable with Phase X / XI numbers. Per the Phase XI card, Phase X acoustic fine-tuning used most of AR_Audios (55 of ~550 clips were excluded), and its 4.5% figure came from a dev set the decoder had been tuned on. Phase XII acoustic training did not include AR_Audios audio (mix: FLEURS Yorรนbรก, Google FLEURS English (en_us), YECS). The evaluations measure different things.
  • Scoring. Not recorded in this card. (The Phase XI card used uppercase + NFC + punctuation removal on both sides.) Whether phonetic_map.json post-processing was applied to the WER figures is not recorded in this card.
  • Not measured: WER by language (Yorรนbรก vs English vs code-switched stretches), tone-mark accuracy, code-switch-point accuracy, named entities, noisy or 8 kHz audio.

Acoustic model

  • Base: facebook/wav2vec2-large-xlsr-53 (Wav2Vec2ForCTC); 424 F32 tensors, 315.5M parameters, 1.26 GB.
  • Frozen: the convolutional feature extractor and the lower 12 transformer layers; per the training script only the upper layers and lm_head were updated.
  • Schedule: 10 epochs, 2,950 steps, wall time 5:47:44, Kaggle multi-GPU, started fresh from the base weights.
  • Data: Google FLEURS Yorรนbรก (yo_ng), Google FLEURS English (en_us), and the YECS Yorรนbรกโ€“English code-switching corpus. Per-source hours and mixing ratios are not recorded in this card.
  • Output head: lm_head.weight [47, 1024]. Before training, the script found pad_id (46) equal to the matrix size (46) and expanded the head by one row. config.json has vocab_size: 47 and pad_token_id: 46.
  • Vocabulary: vocab.json has 46 entries (ids 0โ€“45): [PAD]=0, [UNK]=1, |=2, <s>=40, </s>=41. Added token: <pad>=46 (added_tokens.json).
Step Training loss Validation loss
500 11.866803 5.210579
1000 8.790150 3.593741
1500 6.799892 2.816854
2000 6.284078 2.569384
2500 5.873839 2.456798

Validation loss fell at every logged evaluation (5.21 at step 500 to 2.46 at step 2500), but the gain per 500 steps shrank from 1.62 to 0.11. The run stopped at its fixed 10-epoch budget, not at a stopping criterion; no evaluation is logged at the final step (2,950). The composition of the validation split is not documented here.

Language model and corpus

Master corpus audit. voxira_phaseXII_master_corpus.txt: 381,469 unique normalised lines, including 3,274 with Q, 3,777 with X and 3,219 with Z; 8,274 of them are web-mined sentences collected to cover those letters. Sanitising removed 6,927 web / citation boilerplate lines and capped 28,839 repetitive 3-word Yorรนbรก openings at 100 per prefix, leaving 345,703 clean lines:

Class Lines Share of clean pool
Yorรนbรก 270,862 78.35%
English 42,443 12.28%
Code-switched (YECS) 21,980 6.36%
Unclassified short fragments 10,418 3.01%

Balanced 40 / 30 / 30 training corpus (141,476 lines): Yorรนbรก 56,590 (conversational 4โ€“18-word sentences prioritised), English 42,443 (all clean English, including the Q / X / Z sentences), code-switched 42,443. The code-switched bucket contains the 21,980 unique YECS lines and the 376 AR_metadata training prompts repeated 15ร— (5,640 lines); the remaining 14,823 lines are not itemised in this card.

AR_metadata. 449 rows, 426 unique prompts (23 duplicate rows; AR_277.wav absent). Words per sentence: mean 11.63 (SD 5.81), median 11, range 3โ€“31. Characters: mean 57.26 (SD 29.31), median 54, range 14โ€“154. 72 sentences (16.04%) contain Q (11), X (45) or Z (20). The last 50 unique prompts were held out; the first 376 went into the LM.

KenLM (lmplz -o 4 --prune 0 1 1 2): 75,796 unigram types (1,832,905 tokens); higher orders kept:

Order Kept Total Retained
2-gram 161,969 558,686 28.99%
3-gram 165,990 963,410 17.23%
4-gram 48,550 1,088,627 4.46%

Unigram list (language_model/unigrams.txt): 50,259 words.

Decoding pipeline

  1. Run the acoustic model; take the raw logits.
  2. Fuse the blank rows: logits[:, 0] = np.logaddexp(logits[:, 0], logits[:, 46]).
  3. Mask control rows listed in decoder_config.json ([1, 40, 41, 46]) to -100.
  4. Beam search with pyctcdecode (beam_width=100; alphabet.json uses "" for the blank at row 0 and placeholder symbols for masked rows).
  5. Uppercase, NFC, strip punctuation. phonetic_map.json (50 rules) is optional and not applied in the quickstart: Several rules were written in response to errors seen on the evaluation clips (for example the RABIES / RABBIT case above), so do not apply the map when benchmarking on those clips.

Known limitations

  1. Blank fusion is mandatory (see above). Anything that decodes the raw logits directly, including processor.batch_decode and Wav2Vec2ProcessorWithLM, will emit PAD / [UNK] artefacts and extra spaces.
  2. Phonetic English spellings without the LM. After 10 epochs the raw acoustic output often spells English words phonetically (ACOUNT BAANC, ORNNOUT, HEADAICH, WRITEN, EXAM UESTIONS). Greedy WER is 26.96% on the clean takes. Q, X and Z are in the vocabulary, but no per-letter evaluation exists, and EXAM UESTIONS (for QUESTIONS) shows a dropped Q in greedy output.
  3. High alpha overrides acoustics. Alpha โ‰ฅ 0.60 degraded the probe (see above); the tested range with good results is 0.25โ€“0.35.
  4. KenLM <unk> penalty on zero-count words. A word can be in unigrams.txt (so the prefix trie allows it) yet have no occurrences in the LM text, so it receives unk_score_offset (-10.0). In AR_407.wav an acoustically close in-corpus word (RABIES, seen 3 times) beat RABBIT (zero occurrences) when the audio ended in a sibilant.
  5. Evaluation limits (see Caveats): one small corpus, tuned on the evaluation clips, LM favoured by a matching prompt set, no per-language breakdown.
  6. Data provenance. 8,274 LM sentences were mined from the web, and sources and licences are not itemised here. The code-switched LM bucket contains 14,823 lines whose origin is not itemised. Check the licences of FLEURS, YECS and any web-mined text before commercial use.
  7. Audio scope. 16 kHz mono read / studio speech. Not tested on 8 kHz telephony, overlapping speakers, background music or noise, or Nigerian Pidgin.
  8. No punctuation or casing; output is uppercase. Not a speaker-verification model (do not use transcripts as proof of identity). Human review is advised for legal, medical and financial actions triggered by a transcript.

Quickstart

pip install torch transformers librosa pyctcdecode kenlm huggingface_hub
import re, json, unicodedata
import numpy as np, torch, librosa
from huggingface_hub import snapshot_download
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
from pyctcdecode import BeamSearchDecoderCTC

REPO_ID = "Awesome-x/voxira-yoruba-phaseXII"
device = "cuda" if torch.cuda.is_available() else "cpu"

repo_dir  = snapshot_download(REPO_ID)                      # acoustic weights + language_model/ + decoder files
processor = Wav2Vec2Processor.from_pretrained(repo_dir)
model     = Wav2Vec2ForCTC.from_pretrained(repo_dir).to(device).eval()
decoder   = BeamSearchDecoderCTC.load_from_dir(repo_dir)    # alphabet.json + language_model/ (alpha/beta stored in attrs.json)

with open(f"{repo_dir}/decoder_config.json", encoding="utf-8") as f:
    MASKED_ROWS = json.load(f)["ctc_blank_fusion"]["masked_rows"]

def transcribe(wav_path, beam_width=100):
    audio, _ = librosa.load(wav_path, sr=16000, mono=True)
    inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
    mask = inputs.get("attention_mask")                     # may be absent (return_attention_mask=False)
    with torch.no_grad():
        logits = model(inputs.input_values.to(device),
                       attention_mask=mask.to(device) if mask is not None else None).logits[0].float().cpu().numpy()
    logits[:, 0] = np.logaddexp(logits[:, 0], logits[:, 46])   # REQUIRED: fuse the two blank rows
    for i in MASKED_ROWS:                                      # mask control rows (incl. row 46 after fusion)
        logits[:, i] = -100.0
    text = unicodedata.normalize("NFC", decoder.decode(logits, beam_width=beam_width)).upper()
    text = re.sub(r"[^\w\s]", "", text)
    return re.sub(r"\s+", " ", text).strip()

print(transcribe("sample.wav"))

Tip: split audio longer than ~30 s into chunks before decoding. This exact code was executed against this repository when the card was generated: pyctcdecode 0.5.0, transformers 5.0.0, torch 2.10.0+cu128.

Repository contents and checksums

Path Size SHA-256
added_tokens.json 18 B fc85bf44b931cbd172549d2a7b7eec19276a7900b8c28b00663fe46bcb144f55
alphabet.json 353 B 84ee61564c87e983391b920967501c4d9ea3793afd0eae8a009597d02b887946
config.json 2.3 KB 52680cfdf5ea845f72c995cc7b62b04692bd8447b3bbd32d1ff84f76f0707127
decoder_config.json 1.0 KB a5989b6abf6c22a3595a89bdabb11f2075a9059d4062cfc57faad027f4b572a0
eval_results.json 24.1 KB 6b1e7deb9c32cc6e59e06fd0226aa118216dc3ca4ac074776a4780105efb2c37
language_model/attrs.json 79 B 1e8419cf7292cde4f6c0633065524fd9c67aa95a9e70dd5b271251bc12c6bfde
language_model/phaseXII_40_30_30_prune.bin 11.35 MB 7b9ae0e666f10d4637012accd28787567cb6546c00c605a51aa6f71913b4f4b3
language_model/unigrams.txt 404.2 KB cf52f1839d28525ed44510c09404c2d16169fdbef2aa7eee3bdc111122c72541
model.safetensors 1.26 GB d3917214a410de5379f399ad2629b5ec80041fac57b0d5cb152dee1e57bb0750
optimizer.pt 1.28 GB bee74376f2d6730131f871a295189a18d1e51046d821fbeb5ad1042f9889ec3f
phonetic_map.json 1.8 KB a9d6993b27cb8ecaa1a2a0e001acfb0eaa5cd4c761309cc9fd726f8a07611028
preprocessor_config.json 299 B 8bc3c8d6d97ea61524d5c15d8554b4baf072d2264c0bda1839bc7a822c200d66
rng_state.pth 14.7 KB 15dbe4459d4ef773061c09e46ebef0fde526a9dd25f0f162fd73d243ecd033c3
scaler.pt 1.4 KB 56b8967fdc7656e7c10cf5dc67f14539386e5b031d327d5a70d5f68fbdcd5a80
scheduler.pt 1.5 KB cc59dc5dfd63ab3e3b656944bc3a6031df082267f7df62d6f7e727f91be0a87d
tokenizer_config.json 1.4 KB 15d1f2504bd244010615487c8a003e29831f15de528ea1021a35bb597c384237
trainer_state.json 7.2 KB 63a30846b66112f3727299e880e20ac502f5e3f5ee8b732efab721ffa668d045
vocab.json 529 B 4110c1ddbc5a6322867927977ad98a025b556335b00d4c1387fbb654e1ae0c2f

Reproducibility

Item Value
Acoustic weights model.safetensors, 424 tensors, 315.5M parameters (F32), lm_head.weight [47, 1024]
Training checkpoint checkpoint-2950 (10 epochs, 5:47:44)
Pad ids vocab.json [PAD]=0; config.json pad_token_id=46; <pad>=46 in added_tokens.json
KenLM 4-gram, --prune 0 1 1 2, phaseXII_40_30_30_prune.bin, 10.83 MB
Decoder defaults alpha=0.25, beta=0.5, beam_width=100
Raw numbers eval_results.json in this repo
Environment used to generate this card torch 2.10.0+cu128, transformers 5.0.0, huggingface_hub 1.10.1, pyctcdecode 0.5.0, kenlm 0.3.0
Card generated 2026-11-13 13:54 UTC

Deployment notes

The acoustic model is about 1.26 GB (F32), so cold start is dominated by loading weights; the LM is 10.83 MB. For serverless GPU hosts, bake the weights into the image or a volume rather than downloading on every start. kenlm compiles from source on most hosts. fp16 parity and CPU decoding latency have not been tested.

Not yet done

  • A larger, independent held-out set (and speakers other than the studio recordings).
  • WER broken down by language, tone-mark accuracy, named-entity and domain-term evaluation.
  • 8 kHz telephony and Nigerian Pidgin data.
  • Phase XIII (planned): a longer acoustic run (30 epochs).

Citation

@misc{voxira_yoruba_phaseXII,
  title  = {Voxira Yoruba Phase XII: Yoruba-English code-switching ASR},
  author = {Voxira},
  year   = {2026},
  url    = {https://huggingface.co/Awesome-x/voxira-yoruba-phaseXII}
}
Downloads last month
100
Safetensors
Model size
0.3B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Awesome-x/voxira-yoruba-phaseXII

Finetuned
(408)
this model

Dataset used to train Awesome-x/voxira-yoruba-phaseXII

Evaluation results