whisper-large-v3-turbo-tr-aksan

Yรถresel aฤŸฤฑzlarฤฑ daha iyi anlayan Tรผrkรงe konuลŸma tanฤฑma modeli. DoฤŸu'dan Ege'ye, Karadeniz'den Trakya'ya yรถresel konuลŸmalarฤฑ metne รงevirir. Standart Tรผrkรงedeki baลŸarฤฑsฤฑnฤฑ korur, sessizlikte cรผmle uydurmaz.

Turkish speech recognition tuned for regional dialects. English below.

ร–nce ve sonra / Before and after

Tรผrkรงe

Kฤฑsaca

EฤŸitimsiz Whisper Bu model Ne oldu
Yรถresel aฤŸฤฑzlar %32,5 hata %25,8 hata hatalar beลŸte bir azaldฤฑ
DoฤŸu aฤŸzฤฑ %69,9 hata %46,8 hata hatalar รผรงte bir azaldฤฑ
GรผneydoฤŸu aฤŸzฤฑ %41,9 hata %23,9 hata hatalar neredeyse yarฤฑya indi
Haber ve medya kayฤฑtlarฤฑ %19,0 hata %9,3 hata hatalar yarฤฑya indi
Standart Tรผrkรงe %5,3 hata %5,3 hata aynฤฑ kaldฤฑ, hiรงbir ลŸey kaybedilmedi
Sessizlikte cรผmle uydurma 100 klipte 75 kez 0 kez tamamen ortadan kalktฤฑ

Hata oranฤฑ (WER) nedir?

KonuลŸma tanฤฑma modelleri kelime hata oranฤฑ (Word Error Rate, WER) ile รถlรงรผlรผr: modelin yazdฤฑฤŸฤฑ metin, insanฤฑn dinleyip yazdฤฑฤŸฤฑ doฤŸru metinle kelime kelime karลŸฤฑlaลŸtฤฑrฤฑlฤฑr ve her 100 kelimeden kaรงฤฑnฤฑn hatalฤฑ olduฤŸu sayฤฑlฤฑr. รœรง tรผr hata vardฤฑr:

  • YanlฤฑลŸ kelime: "gidiyoruz" yerine "gidiyoz"
  • Eksik kelime: sรถylenen bir kelimenin hiรง yazฤฑlmamasฤฑ
  • Fazla kelime: sรถylenmeyen bir kelimenin eklenmesi

ร–rnek: 10 kelimelik bir cรผmlede model 2 kelimeyi yanlฤฑลŸ yazdฤฑysa WER %20'dir. DรผลŸรผk olan iyidir; %0 kusursuz demektir.

Bu kartta iki tรผr deฤŸiลŸimden sรถz ediyoruz:

  • Puan farkฤฑ: %32,5'ten %25,8'e inmek 6,7 puanlฤฑk bir dรผลŸรผลŸtรผr.
  • Oransal azalma: aynฤฑ dรผลŸรผลŸ, hatalarฤฑn yaklaลŸฤฑk beลŸte birinin (%21) ortadan kalkmasฤฑ demektir.

Sonuรงlar

1. Yรถresel aฤŸฤฑzlar: kapalฤฑ test

Modelin hiรง gรถrmediฤŸi kayฤฑtlardan oluลŸan bir test seti (holdout) kuruldu: 8 bรถlgeden 92 klip. Bu kayฤฑtlar ve iรงindeki konuลŸmacฤฑlar eฤŸitimde hiรง kullanฤฑlmadฤฑ. Test seti, model seรงimi bittikten sonra yalnฤฑzca bir kez aรงฤฑldฤฑ; yani sonuรง, modelin "ezberlediฤŸi" deฤŸil gerรงekten รถฤŸrendiฤŸi ลŸeyi gรถsterir.

Kapalฤฑ test, bรถlge bรถlge / Holdout by region

Bรถlge EฤŸitimsiz Bu model DeฤŸiลŸim
Tรผm bรถlgeler 32,5 25,8 โˆ’6,7 puan (hatalarฤฑn %21'i gitti)
DoฤŸu 69,9 46,8 โˆ’23,1 puan (%33)
GรผneydoฤŸu 41,9 23,9 โˆ’18,0 puan (%43)
Marmara 36,1 27,2 โˆ’8,9 puan (%25)
Trakya 19,8 15,0 โˆ’4,8 puan (%24)
Karadeniz 27,9 24,3 โˆ’3,6 puan (%13)
Ege 16,8 14,8 โˆ’2,1 puan (%12)
ฤฐรง Anadolu 17,6 18,3 +0,8 puan (deฤŸiลŸmedi)
Akdeniz 21,1 27,5 +6,3 puan (kรถtรผleลŸti)

En bรผyรผk kazanรง, eฤŸitimsiz modelin en รงok zorlandฤฑฤŸฤฑ yerlerde: DoฤŸu ve GรผneydoฤŸu. ฤฐรง Anadolu'da fark yok, Akdeniz'de ise sonuรง kรถtรผleลŸti; bu iki bรถlge daha fazla veriye ihtiyaรง duyuyor (bkz. Amaรง).

ฤฐstatistik ayrฤฑntฤฑlarฤฑ (gรผven aralฤฑklarฤฑ)

KรถลŸeli parantez iรงindeki aralฤฑklar %95 gรผven aralฤฑฤŸฤฑdฤฑr: gerรงek farkฤฑn bรผyรผk olasฤฑlฤฑkla bu aralฤฑkta olduฤŸunu sรถyler. Aynฤฑ kaynak kaydฤฑn klipleri birlikte yeniden รถrneklenerek (bootstrap) hesaplandฤฑ. AralฤฑฤŸฤฑn tamamฤฑ sฤฑfฤฑrฤฑn altฤฑndaysa iyileลŸme tesadรผf deฤŸildir.

Bรถlge Klip Fark [%95 aralฤฑk]
Tรผm bรถlgeler 92 โˆ’6,7 [โˆ’14,9; โˆ’1,0]
Tekrar dรถngรผsรผne giren 5 klip hariรง 87 โˆ’3,0 [โˆ’4,4; โˆ’1,6] (25,2 โ†’ 22,2)
DoฤŸu 13 โˆ’23,1 [โˆ’75,5; โˆ’0,4]
GรผneydoฤŸu 12 โˆ’18,0 [โˆ’27,6; โˆ’7,7]
Marmara 8 โˆ’8,9 [โˆ’11,2; โˆ’5,9]
Trakya 5 โˆ’4,8 [โˆ’6,0; +4,3]
Karadeniz 16 โˆ’3,6 [โˆ’6,1; โˆ’1,2]
Ege 12 โˆ’2,1 [โˆ’4,2; โˆ’0,9]
ฤฐรง Anadolu 14 +0,8 [โˆ’1,3; +6,4]
Akdeniz 12 +6,3 [โˆ’2,4; +15,1]

"Tekrar dรถngรผsรผ": bazฤฑ uzun kliplerde iki model de aynฤฑ kelimeyi defalarca yazdฤฑ. Bu 5 klip รงฤฑkarฤฑldฤฑฤŸฤฑnda da iyileลŸme net biรงimde sรผrรผyor.

2. Standart Tรผrkรงe korundu

Az veriyle yapฤฑlan ince ayarฤฑn bilinen riski, modelin eskiden iyi bildiฤŸi ลŸeyi unutmasฤฑdฤฑr. Google FLEURS standart Tรผrkรงe testinde (300 klip) hata oranฤฑ eฤŸitimden รถnce de sonra da %5,3: hiรงbir ลŸey kaybedilmedi.

3. Haber ve medya kayฤฑtlarฤฑ

EฤŸitimde hiรง kullanฤฑlmayan MediaSpeech haber/medya testinde (200 klip) hata oranฤฑ %19,0'dan %9,3'e indi. Model yalnฤฑzca yรถresel aฤŸฤฑzlarda deฤŸil, gerรงek hayattan gelen genel kayฤฑtlarda da daha iyi.

4. Sessizlikte cรผmle uydurmuyor

Whisper'ฤฑn bilinen bir sorunu, sessizlik ya da gรผrรผltรผ duyunca "ฤฐzlediฤŸiniz iรงin teลŸekkรผrler" gibi hiรง sรถylenmemiลŸ cรผmleler yazmasฤฑdฤฑr. 100 gรผrรผltรผ/sessizlik klibinde:

EฤŸitimsiz Bu model
Bir ลŸey yazdฤฑฤŸฤฑ klip 100 12
Akฤฑcฤฑ, uydurma bir cรผmle yazdฤฑฤŸฤฑ klip 75 0

Kalan 12 klipte de cรผmle deฤŸil, "Tamam" gibi tek bir kฤฑsa kelime yazฤฑyor. DiฤŸer kliplerde yalnฤฑzca bir nokta koyuyor; aลŸaฤŸฤฑdaki kullanฤฑm รถrneฤŸi bunu boลŸ metin sayar.

Amaรง

Bu model bir yapฤฑlabilirlik รงalฤฑลŸmasฤฑdฤฑr. Sorusu ลŸuydu: genel amaรงlฤฑ bir konuลŸma tanฤฑma modeli, az miktarda yรถresel konuลŸmayla, standart Tรผrkรงedeki baลŸarฤฑsฤฑnฤฑ kaybetmeden yรถresel aฤŸฤฑzlarda iyileลŸtirilebilir mi?

Cevap evet: yalnฤฑzca 3,4 saatlik etiketli yรถresel konuลŸmayla (bรถlge baลŸฤฑna yarฤฑm saat ile bir saat arasฤฑ) hatalar beลŸte bir azaldฤฑ. Bu bir baลŸlangฤฑรง noktasฤฑdฤฑr. Daha รงok yรถresel kayฤฑt, daha รงok farklฤฑ konuลŸmacฤฑ ve รถzenle doฤŸrulanmฤฑลŸ etiketlerle รงok daha iyi sonuรงlar alฤฑnmasฤฑ beklenir. Ek verinin en รงok fayda saฤŸlayacaฤŸฤฑ yerler hรขlรข en zayฤฑf kalan bรถlgelerdir: DoฤŸu, ฤฐรง Anadolu ve Akdeniz.

Nasฤฑl kullanฤฑlฤฑr

from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="eqlx-ai/whisper-large-v3-turbo-tr-aksan",
               torch_dtype="auto", device="cuda")  # GPU yoksa device="cpu"
out = asr("kayit.wav", chunk_length_s=30, batch_size=8,
          generate_kwargs={"language": "turkish", "task": "transcribe"})
text = out["text"].strip()
if not any(ch.isalnum() for ch in text):  # sessizlikte model bazen yalnฤฑzca "." yazar
    text = ""

Uzun kayฤฑtlarda รถnรผne bir ses etkinliฤŸi algฤฑlayฤฑcฤฑsฤฑ (VAD, รถr. Silero VAD) koymak รถnerilir: konuลŸma olmayan bรถlรผmler hiรง modele gitmez.

Model bilgisi. Temel model openai/whisper-large-v3-turbo ยท 809M parametre ยท fp16, 1,6 GB ยท Lisans CC-BY-NC-4.0 ยท Ek aฤŸฤฑrlฤฑklar ana modele birleลŸtirildi, peft gerekmez.

Nasฤฑl eฤŸitildi

Yรถntem: LoRA ile ince ayar. Ana modelin 815 milyon aฤŸฤฑrlฤฑฤŸฤฑ olduฤŸu gibi bฤฑrakฤฑldฤฑ; yanฤฑna yalnฤฑzca 6,5 milyon (%0,8) yeni, kรผรงรผk aฤŸฤฑrlฤฑk eklenip bunlar eฤŸitildi (LoRA, Low-Rank Adaptation). Bu yรถntem az veriyle รงalฤฑลŸฤฑrken modelin eski bilgisini korur. EฤŸitim bitince bu ek aฤŸฤฑrlฤฑklar ana modelin iรงine birleลŸtirildi.

Veri: toplam 6,4 saat, 1536 klip.

Parรงa Sรผre Klip Neden
Yรถresel konuลŸma (rรถportaj, sohbet) 3,4 saat 587 yรถresel aฤŸฤฑzlarฤฑ รถฤŸrenmek
Google FLEURS standart Tรผrkรงe (CC-BY-4.0) 2,5 saat 749 standart Tรผrkรงeyi unutmamak
รœretilmiลŸ gรผrรผltรผ ve sessizlik, boลŸ cevaplฤฑ 0,5 saat 200 sessizlikte cรผmle uydurmamak

Yรถresel konuลŸmanฤฑn bรถlgelere daฤŸฤฑlฤฑmฤฑ (saat): DoฤŸu 0,90 ยท Ege 0,71 ยท ฤฐรง Anadolu 0,49 ยท Trakya 0,37 ยท Akdeniz 0,34 ยท Karadeniz 0,31 ยท GรผneydoฤŸu 0,25. Kayฤฑtlar metne dรถkรผldรผ (etiketlendi); ลŸรผpheli klipler insan tarafฤฑndan dinlenip dรผzeltildi. KonuลŸmanฤฑn รงok seyrek olduฤŸu klipler (mรผzik, uzun sessizlik) รงฤฑkarฤฑldฤฑ. Test kayฤฑtlarฤฑ ve konuลŸmacฤฑlarฤฑ eฤŸitimden tamamen ayrฤฑ tutuldu; test cรผmlelerinin hiรงbiri eฤŸitim metinlerinde geรงmiyor. Ses dosyalarฤฑ bu depoda yayฤฑnlanmaz.

EฤŸitim seyri. Kayฤฑp (loss: modelin doฤŸru metinden ne kadar uzak olduฤŸunu gรถsteren sayฤฑ) ilk turda hฤฑzla dรผลŸtรผ, sonra dengelendi.

EฤŸitim kaybฤฑ / Training loss

EฤŸitim boyunca her 50 adฤฑmda modelin bir kopyasฤฑ kaydedildi ve ayrฤฑ bir geliลŸtirme setinde รถlรงรผldรผ. Yรถresel hata adฤฑm adฤฑm %29'dan %20'ye indi; sessizlikte bir ลŸey yazma oranฤฑ %100'den %12'ye dรผลŸtรผ. En iyi sonuรง veren son kopya seรงildi.

Kopyalar adฤฑm adฤฑm / Checkpoints

Teknik ayarlar
Ayar DeฤŸer
LoRA rank / alpha / dropout 32 / 16 / 0,05
LoRA hedefleri q_proj, v_proj, kodlayฤฑcฤฑ (encoder) ve รงรถzรผcรผde (decoder)
EฤŸitilen parametre 6,55M / 815M (%0,80)
Optimizasyon AdamW, aฤŸฤฑrlฤฑk azaltma 0, gradyan kฤฑrpma 1,0
ร–ฤŸrenme hฤฑzฤฑ 1e-4 tepe; ilk %10 (29 adฤฑm) doฤŸrusal ฤฑsฤฑnma, sonra doฤŸrusal azalma
Etkin batch 16 (8 ร— 2 gradyan biriktirme)
Tur / adฤฑm 3 / 288
Hassasiyet bf16 autocast, gradient checkpointing
Kayฤฑp token รงapraz entropisi, รถnce her klip iรงinde, sonra klipler arasฤฑnda ortalanฤฑr (kฤฑsa ve boลŸ cevaplฤฑ klipler uzun kliplerle eลŸit aฤŸฤฑrlฤฑk alฤฑr)
Hedef metin <|tr|><|transcribe|><|notimestamps|> + metin + <|endoftext|>; en fazla 440 token
Klipler ortalama 21 sn (4โ€“25 sn); saniyede 1 kelimenin altฤฑndaki klipler รงฤฑkarฤฑldฤฑ
Kopya seรงimi her 50 adฤฑmda bir kopya, geliลŸtirme setinde (141 klip) WER ile รถlรงรผldรผ; adฤฑm 288 seรงildi
Donanฤฑm tek NVIDIA RTX 4090 (24 GB), en yรผksek bellek 6,7 GB

Lisans

CC-BY-NC-4.0: araลŸtฤฑrma ve kiลŸisel kullanฤฑm serbest, ticari kullanฤฑm yok. Modeli indirip deneyebilir, รผzerinde รงalฤฑลŸabilir ve paylaลŸabilirsiniz; paylaลŸฤฑrken bu depoya kaynak gรถsterin. รœcretli bir รผrรผnde veya hizmette kullanmak iรงin izin gerekir. Bu model bir yapฤฑlabilirlik รงalฤฑลŸmasฤฑ olarak yayฤฑnlanmฤฑลŸtฤฑr. Temel model Whisper MIT lisanslฤฑdฤฑr; standart Tรผrkรงe verisi Google FLEURS'tรผr (CC-BY-4.0).

Sฤฑnฤฑrlar

  • Test setleri kรผรงรผk. Bรถlge baลŸฤฑna 5โ€“16 klip; bรถlge bรถlge sonuรงlar kaba bir gรถstergedir.
  • Her bรถlgede kazanรง yok. ฤฐรง Anadolu'da deฤŸiลŸim yok, Akdeniz'de kรถtรผleลŸme var; DoฤŸu'da hata hรขlรข yรผksek.
  • Tekrar dรถngรผsรผ. Uzun veya ลŸarkฤฑya benzeyen kliplerde model ara sฤฑra aynฤฑ kelimeyi tekrar tekrar yazabilir (kapalฤฑ testte 92 klipten 5'i, eฤŸitimsiz modelde de gรถrรผlรผyor).
  • Yazฤฑm stili. Test metinleri eฤŸitim metinleriyle aynฤฑ sรผreรงten geรงti; kazancฤฑn bir kฤฑsmฤฑ noktalama ve sayฤฑ yazฤฑmฤฑ gibi stile uyumdan gelebilir.
  • Kapsam dฤฑลŸฤฑ. Kรผrtรงe, Zazaca, Arapรงa ve diฤŸer diller; kimin konuลŸtuฤŸunu ayฤฑrma; zaman damgasฤฑ. Tฤฑbbi, hukuki veya kimlik doฤŸrulama gibi hatanฤฑn zarar vereceฤŸi kullanฤฑmlar iรงin tek baลŸฤฑna uygun deฤŸildir.

English

Turkish speech recognition tuned for regional dialects. Fine-tuned from openai/whisper-large-v3-turbo on 3.4 h of regional Turkish speech. It keeps its accuracy on standard Turkish and no longer invents sentences on silence.

At a glance

Base Whisper This model Change
Regional dialects (holdout) 32.5% WER 25.8% WER one fifth of errors gone
Eastern 69.9% 46.8% one third fewer errors
South-Eastern 41.9% 23.9% almost half the errors
News and media (MediaSpeech) 19.0% 9.3% errors halved
Standard Turkish (FLEURS) 5.3% 5.3% unchanged
Invented sentences on silence 75 of 100 clips 0 gone

What is WER?

Word error rate: the model's transcript is compared word by word with a human reference, and wrong, missing and extra words are counted per 100 words. 2 wrong words in a 10-word sentence is 20% WER. Lower is better.

Results

Holdout, regional dialects. 92 clips from 8 regions; no recording or speaker was used in training; the set was opened once, after the model was chosen. The per-region chart and table are in the Turkish section.

Region Base This model Change [95% CI]
All 32.5 25.8 โˆ’6.7 [โˆ’14.9; โˆ’1.0]
All, without 5 clips where either model loops 25.2 22.2 โˆ’3.0 [โˆ’4.4; โˆ’1.6]
Eastern 69.9 46.8 โˆ’23.1 [โˆ’75.5; โˆ’0.4]
South-Eastern 41.9 23.9 โˆ’18.0 [โˆ’27.6; โˆ’7.7]
Marmara 36.1 27.2 โˆ’8.9 [โˆ’11.2; โˆ’5.9]
Thrace 19.8 15.0 โˆ’4.8 [โˆ’6.0; +4.3]
Black Sea 27.9 24.3 โˆ’3.6 [โˆ’6.1; โˆ’1.2]
Aegean 16.8 14.8 โˆ’2.1 [โˆ’4.2; โˆ’0.9]
Central Anatolia 17.6 18.3 +0.8 [โˆ’1.3; +6.4]
Mediterranean 21.1 27.5 +6.3 [โˆ’2.4; +15.1]

95% intervals from a bootstrap over source recordings. Other tests: FLEURS tr test (300 clips) 5.3 โ†’ 5.3; MediaSpeech tr (200 clips) 19.0 โ†’ 9.3; noise/silence (100 clips): any output 100 โ†’ 12, fluent invented sentence 75 โ†’ 0.

Purpose

A feasibility study: with only 3.4 hours of labelled regional speech (half an hour to an hour per region), a general-purpose recogniser improves clearly on regional dialects without losing standard Turkish. More regional recordings, more speakers and carefully verified labels should give much better results, most of all in the weakest regions (Eastern, Central Anatolia, Mediterranean).

How to run

Use the Python snippet in the Turkish section: the transformers ASR pipeline with language="turkish" and chunk_length_s=30; treat a punctuation-only output as empty; put a VAD (e.g. Silero VAD) in front of long recordings.

Training

LoRA fine-tune (6.55M of 815M parameters trained, 0.80%), merged into the base weights; no peft needed. Data: 6.4 h in 1536 clips: 3.4 h regional speech (587 clips; Eastern 0.90, Aegean 0.71, Central Anatolia 0.49, Thrace 0.37, Mediterranean 0.34, Black Sea 0.31, South-Eastern 0.25 h), transcribed with human review of suspicious clips; 2.5 h Google FLEURS tr train (CC-BY-4.0) so standard Turkish is not forgotten; 0.5 h generated noise/silence with empty targets. Test recordings and speakers are excluded from training. Audio is not redistributed. Full settings are in the "Teknik ayarlar" table above; one NVIDIA RTX 4090.

Licence

CC-BY-NC-4.0: free for research and personal use, no commercial use. You may use, adapt and share the model with attribution to this repository; use in a paid product or service needs permission. Released as a feasibility study. The base model Whisper is MIT; the standard Turkish data is Google FLEURS (CC-BY-4.0).

Limitations

Small test sets (5โ€“16 clips per region). No gain in Central Anatolia, a loss in the Mediterranean; Eastern is still hard. Occasional repetition loops on long or song-like audio. Test references went through the same labelling process as training labels, so part of the gain may be label-style match. Out of scope: Kurdish, Zaza, Arabic and other languages; speaker diarization; timestamps; high-stakes uses (medical, legal, identity).

About

Built by eqlx. How we fine-tune speech models on an institution's own audio and terms: small language models ยท contact center speech use case.

Downloads last month
66
Safetensors
Model size
0.8B params
Tensor type
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for eqlx-ai/whisper-large-v3-turbo-tr-aksan

Adapter
(177)
this model

Evaluation results

  • regional WER (untuned base 32.5) on tr-stt-aksan regional holdout (92 clips
    self-reported
    25.800
  • WER (untuned base 5.3) on google/fleurs tr_tr test (300 clips)
    self-reported
    5.300