Request access to Kashgari Large 1.0

Research evaluation only. Each request is reviewed by hand.

The weights are shared for research evaluation under the terms in LICENSE: no commercial use, no redistribution, no training on the model or its outputs, and publishing scores needs the author's written permission, except on public speech recognition leaderboards. Hugging Face shares your username and email with the author, who uses them only to manage access.

Log in or Sign Up to review the conditions and access this model content.

Kashgari Large 1.0

Kashgari Large 1.0 is a Turkish speech recognition model for research. It is whisper-large-v3 fine-tuned on 4,737 h of Turkish speech, including parliament recordings and audiobooks, and it is the most accurate model of this project. Part of its training data is non-commercial or not cleared for reuse, so it may be used for research evaluation only.

Model details

Developer Cagan Oflazoglu
Model type Whisper encoder-decoder, 32 + 32 layers, 1.54B parameters
Language Turkish
Fine-tuned from openai/whisper-large-v3
Formats transformers (float16) and CTranslate2 for faster-whisper
Licence research evaluation terms (LICENSE), no commercial use
Version 1.0, tag r1.0, November 2026
Commercial sibling Bitig Large 1.0, trained only on commercially licensed data

Results

Word error rate (WER, %) after the normalizer tr-norm-v1 (lowercase, punctuation removed, numbers written as words), decoded with CTranslate2 in float16, beam 5, no timestamps.

Test sets

I scored the test sets once, on 7 November 2026, with the model, the decoding settings and the normalizer fixed before the run, and scored whisper-large-v3 again in the same run. The difference row is Kashgari's WER minus whisper-large-v3's, with a 95 % interval from a paired bootstrap (10,000 resamples; FLEURS resampled by sentence, since most sentences are read two or three times, and parliament by session).

FLEURS tr test (743 clips) Common Voice 17 tr test (2,000) Parliament, 10 sessions (1,000) MediaSpeech tr (1,000)
Kashgari Large 1.0 4.53 5.39 6.60 11.28
whisper-large-v3 6.05 7.90 9.59 11.21
Difference -1.53 [-1.98, -1.09] -2.51 [-3.05, -2.01] -2.99 [-3.31, -2.61] +0.07 [-0.58, +0.75]

The parliament test set is ten whole sessions that Kashgari never trained on, cut from WorldSpeech's test and train splits together. MediaSpeech is broadcast media, and there Kashgari is no better than whisper-large-v3: its gains come from read speech and parliament, and they do not carry over to that domain.

An audio fingerprint check of the training audio against every test set found one of the 1,000 MediaSpeech test clips (998a8657-04ec-4c69-9328-c4bec9ab57bb.flac) inside the ISSAI corpus, which Kashgari trained on. Without it Kashgari scores 11.27 and whisper-large-v3 11.21, so the figures above stand. The check finds about 6 in 10 degraded copies, so it can miss some. In the training audio checked so far (ISSAI, and the TBMM and KKTC parliament clips), no copy of a FLEURS, Common Voice or parliament test clip was found. The WorldSpeech, Common Voice 17, FLEURS, Khan Academy and KIRAAT training audio is not checked yet.

Who said what: TR-Diar v1

TR-Diar v1 is the project's meeting set: six Turkish parliament sittings cut into 133 ten-minute windows (22.17 h), with speaker turns aligned to the official record. Each system pairs a transcription model with a speaker model, and cpWER counts the words attributed to the wrong speaker as errors too. Scored once, under rules written before any of these numbers existed:

Pipeline cpWER WER, speakers ignored
Kashgari Large 1.0 + D1 7.63 6.18
whisper-large-v3 + DiariZen, dotted-I and filler steps (best whisper-large-v3 pipeline) 9.02 7.44
whisper-large-v3 + D1 9.28 7.73
whisper-large-v3 + pyannote community-1 9.46 7.73
whisper-large-v3 + Nemotron-3-Diarization 14.45 7.73

Kashgari with D1 minus the best whisper-large-v3 pipeline: -1.39 [-1.63, -1.14] (paired bootstrap over windows, 10,000 resamples). D1 is the project's research speaker model (pyannote community-1 with fine-tuned segmentation).

Dev sets

FLEURS tr (338 clips) Common Voice 17 tr (2,000) Parliament (1,000)
Kashgari Large 1.0 5.27 4.56 5.67
Bitig Large 1.0 5.94 5.15 8.57
whisper-large-v3 7.48 6.71 8.72

Parliament is five whole sessions of the Turkish parliament, from WorldSpeech, that neither model trained on.

Phone and room audio, FLEURS tr dev through simulated channels:

Channel whisper-large-v3 Kashgari Large 1.0
Landline (G.711 A-law, 8 kHz) 10.57 7.28
Mobile (AMR narrowband, 7.4 kbit/s) 11.60 8.29
Internet call (Opus, 12 kbit/s) 8.36 6.04
Far room (reverberation, pink noise at 10 to 15 dB SNR) 25.77 17.16

Usage

With faster-whisper and the CTranslate2 build in ct2/:

from faster_whisper import WhisperModel

model = WhisperModel("ct2", device="cuda", compute_type="float16")  # "int8" on a CPU
segments, _ = model.transcribe("audio.wav", language="tr", beam_size=5, without_timestamps=True)
print(" ".join(s.text.strip() for s in segments))

With transformers:

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition", model="turkish-sa-asr/kashgari-large-v1.0", dtype="float16", device="cuda"
)
print(asr("audio.wav", generate_kwargs={"language": "tr", "task": "transcribe", "num_beams": 5})["text"])

Every score on this card comes from the CTranslate2 build. On the dev sets, transformers with the settings above scored 0.15 points higher WER on FLEURS (95% interval [−0.06, +0.38]) and 0.05 lower on Common Voice 17 ([−0.24, +0.14]) (results/eval/backend_equiv.json).

For recordings longer than 30 s, use voice activity detection (eval/transcribe.py --long does).

Evaluation

eval/ reproduces the numbers above:

pip install -r eval/requirements.txt
python eval/smoke.py .                              # FLEURS tr dev: prints WER 5.27 on a CUDA GPU in float16
python eval/transcribe.py . clips/ hyps.jsonl        # decodes a folder with the settings above
python eval/score.py refs.jsonl hyps.jsonl [--raw]   # corpus WER and CER after tr-norm-v1

For a fair comparison, score with eval/normalize.py, or report raw WER next to your own normalizer: Whisper's English normalizer maps I to i instead of ı. Please report the tag (r1.0) or the commit hash with your scores.

Kashgari trained on the train splits of FLEURS, Common Voice 17 and the ISSAI Turkish Speech Corpus, never on their dev or test splits, and on no other Common Voice release. Three things to check before scoring:

  • WorldSpeech's own test split: do not use it. It shares 188 of its 197 sessions with the train split Kashgari trained on; the parliament dev set above is cut from whole held-out sessions instead.
  • Recordings of the Turkish parliament (TBMM) or the KKTC Assembly: Kashgari trained on them; I can share the session list.
  • Common Voice 18 or later: check the test split against CV17's train split, since releases can re-split clips.

Training data

Source Hours Licence
KIRAAT audiobooks (serdarcaglar/kiraat), transcripts written by whisper-large-v3 2,506.83 none declared
KKTC Assembly recordings 946.07 public proceedings, not cleared
WorldSpeech tr, parliament, train split minus 15 held-out sessions 655.49 CC BY-NC 4.0
TBMM plenary recordings, segments not in WorldSpeech 319.18 public proceedings, not cleared
ISSAI Turkish Speech Corpus, train 209.32 MIT
Khan Academy Turkish 73.61 CC BY-SA 3.0
Common Voice 17 tr, train, seen twice per epoch 19.82 CC0-1.0
FLEURS tr, train, seen twice per epoch 8.28 CC BY 4.0

Parliament segments were kept where the official transcript agreed with a Whisper reading of the audio (character error rate up to 0.15). WorldSpeech sentences sharing an 8-word sequence with a dev or test text were dropped.

Training ran on a Kaggle TPU v5e-8, with room and noise augmentation from OpenSLR 28:

  1. A1: 2 epochs over the five human-transcribed sources (966.49 h), learning rate 1e-5, batch 64.
  2. A2: from A1, 1 epoch over all eight sources (4,737.49 h), learning rate 1e-5, batch 64.
  3. Kashgari Large 1.0 is the average of the A1 and A2 weights, a model soup (Wortsman et al., 2022).

Intended use and limitations

Kashgari is meant for research on spoken Turkish: benchmarking, linguistics, and studies of parliament and media speech. It may not be used commercially.

  • Parliament minutes are edited, not verbatim, so the model may drop repetitions and hesitations.
  • On broadcast media (MediaSpeech) it is no more accurate than whisper-large-v3.
  • Real phone calls and meetings have no Turkish test set here yet; the channel results are simulations.
  • Like Whisper, it can write stock phrases over silence.

License

Research evaluation only, under the terms in LICENSE, accepted in the access form above. Public speech recognition leaderboards may evaluate the model and publish its scores; anyone else needs my written permission first.

Citation

@misc{oflazoglu2026kashgari,
  author = {Oflazoglu, Cagan},
  title  = {Kashgari Large 1.0},
  year   = {2026},
  url    = {https://huggingface.co/turkish-sa-asr/kashgari-large-v1.0}
}

References

Asonitis, A., Lanzendörfer, L. A., Berdoz, F., & Wattenhofer, R. (2026). WorldSpeech: A multilingual speech corpus from around the world (arXiv:2605.09167). arXiv. https://arxiv.org/abs/2605.09167

Conneau, A., Ma, M., Khanuja, S., Zhang, Y., Axelrod, V., Dalmia, S., Riesa, J., Rivera, C., & Bapna, A. (2023). FLEURS: Few-shot learning evaluation of universal representations of speech. 2022 IEEE Spoken Language Technology Workshop (SLT), 798-805.

Mussakhojayeva, S., Dauletbek, K., Yeshpanov, R., & Varol, H. A. (2023). Multilingual speech recognition for Turkic languages. Information, 14(2), 74.

Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2023). Robust speech recognition via large-scale weak supervision. Proceedings of the 40th International Conference on Machine Learning, 28492-28518.

serdarcaglar. (2026). KIRAAT: A Turkish read-speech corpus [Dataset]. Hugging Face. https://huggingface.co/datasets/serdarcaglar/kiraat

Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., & Schmidt, L. (2022). Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. Proceedings of the 39th International Conference on Machine Learning, 23965-23998.

Downloads last month
15
Safetensors
Model size
2B params
Tensor type
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for turkish-sa-asr/kashgari-large-v1.0

Finetuned
(1,110)
this model
Quantizations
1 model

Datasets used to train turkish-sa-asr/kashgari-large-v1.0

Collection including turkish-sa-asr/kashgari-large-v1.0

Paper for turkish-sa-asr/kashgari-large-v1.0

Evaluation results