Bengali speech recognition works on read speech and breaks on the dialects people speak.

Where the models stand

Vistaar scored public and commercial systems on five Bengali test sets. All five are read speech.^[1]

No single system wins every column. Azure is best on Common Voice and worst of the four on FLEURS. Vistaar has no Bengali telephone or conversational set, so this table says nothing about phone calls.^[1]

Zero-shot Whisper does far worse. Whisper-large-v2 scored 104.1% on Bengali FLEURS and 103.4% on Common Voice 9.^[2] A WER above 100% means the output had more errors than the reference had words. Whisper’s training data included 1.3 hours of transcribed Bengali, against 12 hours of Hindi and 104 of Urdu.^[2]

Newer commercial systems are much better, but still far from fine-tuned ones. On IndicVoices, a larger and more varied Bengali test set, Gemini-2.5-Pro scored 22.2%, Sarvam’s Saarika 25.2%, Azure 34.8% and Meta’s MMS 47.2%. A Canary model fine-tuned on Indian speech scored 11.2%.^[3]

One language, many dialects

Most Bengali test sets use standard Bengali. Regional dialects differ from it in sounds, sentence structure and vocabulary, and Ben-10’s authors document large differences for every region they recorded.^[4]

Ben-10 is a 78-hour corpus of spontaneous speech from ten regions of Bangladesh, recorded from 394 speakers.^[4] Its authors scored seven systems on each region. Zero-shot Whisper-large-v3 averaged 113% WER, and Google’s speech API 103%.^[4] Tugstugi, a Whisper model fine-tuned on standard Bengali that won an earlier Bengali ASR competition, averaged 81%. Fine-tuning it further on Ben-10 brought the average to 70%.^[4]

The spread between regions is wide. The fine-tuned model scored 36% on Tangail and 85% on Chittagong.^[4] The authors link this to vocabulary: about 62% of the dialect words in Sandwip and 43% in Tangail do not exist in standard Bengali.^[4] A model trained only on standard Bengali found 1.4 to 9% of the dialect words, mapping the rest to the nearest standard word. After fine-tuning on Ben-10, it found 16 to 53%.^[4]

A second study used RegSpeech12, a test set from 12 regions. Whisper-medium fine-tuned on standard Bengali scored 84.6% WER and a dialect-trained model 74.1%.^[5] Comilla was the easiest region at 43.8%, and Sylhet was among the hardest at 83.1%.^[5]

If your users speak a regional dialect, a FLEURS or Kathbath score will not predict how a model does for them.

Drama, sermons and telemedicine

Read speech is the easy case. OOD-Speech built a test set from 17 kinds of real Bengali audio: TV drama, audiobooks, talk shows, parliament sessions, online classes, advertisements, Islamic sermons and more.^[6]

Both baseline systems did worse there than on a crowdsourced test set. Google’s API scored 41% WER on the crowdsourced set, and 49 to 89% across the 17 domains.^[6] A wav2vec 2.0 model fine-tuned on Bengali went from 38% to between 42 and 80%.^[6] Background music, crowd noise and fast speech explain part of it. Sermons add frequent Arabic, Persian and Urdu words.^[6]

The study also included 0.73 hours of telemedicine audio full of medicine names, symptoms and treatments.^[6] Google scored 73% and the fine-tuned wav2vec 2.0 model 72% on it.^[6] That is the only Bengali medical speech result we could find.

Long recordings add their own problems. Bengali-Loop collected 158.6 hours of long-form audio from 11 YouTube channels, mostly Bangla drama.^[7] Of the two models it tested, a Whisper-medium fine-tuned on Bengali scored 34.1% WER and a second Bengali model 50.7%.^[7]

The clinical pattern from other languages also applies. On accented clinical English from Africa, Afonja et al. found that models with low overall WER still made more errors on drug names, conditions and lab results, and that fine-tuning on clinical speech cut medical WER by 25 to 34% relative.^[11]

English words in Bengali sentences

Spoken Bengali borrows English words, as in the example above. A transcriber has to choose a script for each one, and so does a model.

The MUCS 2021 challenge released 46 hours of Bengali-English speech from computer science tutorials.^[8] In its blind test set, 33.6% of words were English.^[8] English words caused 55 to 58% of all substitution and deletion errors, depending on the system.^[8] The organisers also found the same English word written in both Latin and Bengali script in the training transcripts, and words that mixed the two inside a single word, which they found only in the Bengali-English data.^[8]

They added a transliterated WER (T-WER) that accepts an English word written in either script. It lowered the baseline systems’ Bengali-English scores by 1.0 to 1.8 points.^[8] Even then, the baselines scored 31 to 36%.^[8]

Before you compare models on Bengali with English in it, decide which script a correct English word uses, and score both scripts as correct if you accept both.

One word, several correct spellings

Bengali spelling allows variation, and Unicode allows more. The letter য় can be stored as one code point (U+09DF) or as য plus a nukta sign (U+09AF U+09BC). They look identical on screen. A scorer that compares raw strings counts them as different words.

Ansary et al. ran a Bengali normaliser over unique words from the OSCAR web corpus. It flagged 369,348 of 2,883,731 unique Bengali words, or 12.8%, as needing normalisation.^[9]

Spelling variants matter even after Unicode is fixed. A 2026 study scored Bengali IndicVoices against a list of valid spellings for each word instead of a single reference.^[3] Every system’s error rate fell. Gemini-2.5-Pro went from 22.2% to 12.8%, Saarika from 25.2% to 13.8% and Azure from 34.8% to 21.0%.^[3] The fine-tuned Canary model only went from 11.2% to 7.7%.^[3] So part of the gap between commercial and fine-tuned models comes from spellings a reader would accept.

The normaliser that hides vowel errors

Whisper’s default normaliser for non-English text replaces every Unicode combining mark with a space.^[10] In Bengali, that includes every vowel sign and the hasant. Manohar et al. showed that this rule makes Indic results look better than they are, and they name Bengali among the affected languages, though they report numbers only for Hindi, Tamil and Malayalam.^[10]

We ran the rule on a few Bengali words to see what it does. দিন (“day”) and দীন (“poor”) both become দ ন, so a model that writes the wrong word is scored as correct. কাল (“tomorrow”), কিল (“punch”) and কুল (“shore”) all become ক ল. ডাক্তার (“doctor”) becomes four fragments, ড ক ত র.

When you read a Bengali WER, check which normaliser produced it. If it is Whisper’s default, vowel errors may not be counted at all.

What helps

• Test on speech like your users’. If they speak a regional dialect, use Ben-10 or RegSpeech12 for that region, not FLEURS.^{[4, 5]}
• Fine-tune on dialect data. On Ben-10, fine-tuning cut average WER from 81% to 70% and raised dialect-word recall from under 10% to as much as 53%.^[4]
• Fine-tune on Bengali at all. IndicWhisper scored 20.9% on FLEURS where zero-shot Whisper-large-v2 scored 104.1%.^{[1, 2]}
• Apply NFC Unicode normalisation, or a Bengali normaliser, to both reference and output before scoring.^[9]
• Don’t score Bengali with Whisper’s default normaliser. Use a rule set that keeps vowel signs and the hasant.^[10]
• Write a script policy for English words, and accept both scripts if you can (T-WER).^[8]
• Allow spelling variants. Scoring against valid spellings removed 4 to 14 points of error on Bengali.^[3]
• Confirm critical fields. For a dosage, an account number or an address, ask the speaker to confirm instead of trusting the transcript.

Open gaps

• A Bengali clinical speech test set with consent, de-identification and clinician-checked references. The 0.73-hour telemedicine subset is the only one we found.
• Bengali as spoken in West Bengal and the rest of India. The dialect corpora we found cover Bangladesh only.
• Telephone and call-centre Bengali results for current commercial models.
• Studies of names, numbers and addresses in Bengali speech recognition. We could not find one.
• Zero-shot results for Whisper-large-v3 and current commercial models on standard read Bengali, scored the same way.

We plan to cover several of these in QuantClarity’s own Bengali evaluations.

References

1. K. S. Bhogale, S. Sundaresan, A. Raman, T. Javed, M. M. Khapra, P. Kumar. Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR. Interspeech 2023.
2. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, I. Sutskever. Robust Speech Recognition via Large-Scale Weak Supervision. ICML 2023.
3. K. S. Bhogale, T. Javed, G. S. John, D. Rathi, A. Padmanaban, N. Parasa, M. M. Khapra. Towards Orthographically-Informed Evaluation of Speech Recognition Systems for Indian Languages. ICASSP 2026.
4. T. T. Dipto, A. Hossain, R. S. Faruque, et al.. Are ASR foundation models generalized enough to capture features of regional dialects for low-resource languages?. IJCNLP-AACL 2025.
5. J. Hasan, S. R. Dipta. BanglaTalk: Towards Real-Time Speech Assistance for Bengali Regional Dialects. arXiv preprint, 2025.
6. F. R. Rakib, S. S. Dip, S. Alam, et al.. OOD-Speech: A Large Bengali Speech Recognition Dataset for Out-of-Distribution Benchmarking. arXiv preprint, 2023.
7. H. M. S. Tabib et al.. Bengali-Loop: Community Benchmarks for Long-Form Bangla ASR and Speaker Diarization. arXiv preprint, 2026.
8. A. Diwan, R. Vaideeswaran, S. Shah, et al.. MUCS 2021: Multilingual and Code-Switching ASR Challenges for Low Resource Indian Languages. Interspeech 2021.
9. N. Ansary, Q. A. R. Adib, T. Reasat, et al.. Unicode Normalization and Grapheme Parsing of Indic Languages. LREC-COLING 2024.
10. K. Manohar, L. G. Pillai, E. Sherly. What is lost in Normalization? Exploring Pitfalls in Multilingual ASR Model Evaluations. EMNLP 2024.
11. T. Afonja, T. Olatunji, S. Ogun, N. A. Etori, A. Owodunni, M. Yekini. Performant ASR Models for Medical Entities in Accented Speech. Interspeech 2024.