Urdu Sukhan ASR

Results

Words wrong (WER, %) after the same normalisation for both models, with greedy decoding in Hugging Face Transformers. None of these sets was used in training.

The test sets we can share, 808 clips (about 96 minutes) from the Common Voice and FLEURS Urdu test sets and eight stress sets, are on Hugging Face on request. Get IndicVoices, UrduSpeech and nzg73 from their publishers.

Test sets on Hugging FaceResearch report

Compared with Whisper large-v3

Urdu Sukhan ASR has the layout of Whisper large-v3-turbo: the same 32-layer encoder as Whisper large-v3 and a 4-layer decoder instead of 32.

OpenAI’s speed figure comes from the smaller decoder. On a CPU in whisper.cpp, the encoder, which both models share in size, does about 95% of the work, so the gain there is much smaller. We have not timed or scored Whisper large-v3 ourselves.

Sources:

OpenAI Whisperwhisper.cpp models

Use it

Try it in Colab, without installing

Our Colab notebook runs the model in your own Google Colab session, on a free GPU or the CPU: record from the microphone, upload a file or try the example clips, with the recommended settings built in. It needs an approved access request and a free Google account. Your audio goes to your Colab session on Google’s servers, never to Loudness.ai.

Try it in Colab

whisper.cpp (command line)

git clone https://github.com/Loudness-ai/urdu-sukhan-asr.git
cd urdu-sukhan-asr
bash examples/whisper_cpp.sh speech.wav

faster-whisper (Python)

python -m pip install faster-whisper==1.2.1 ctranslate2==4.8.3 huggingface_hub==2.2.0 soundfile==0.14.0 soxr==1.1.0
python examples/transcribe_faster_whisper.py speech.wav

Transformers (Python)

python -m pip install -r examples/requirements.txt
python examples/transcribe_transformers.py speech.wav

The model files are shared on request, so first request access on Hugging Face and log in with hf auth login. Each command then runs a tested script from the examples folder of the GitHub repository. The installation guide covers each format step by step, with the settings and common problems:

Recommended settings

• Set the language (ur or en) when you know it. Auto-detection runs the encoder an extra time in whisper.cpp.
• Turn off blank suppression so that silence returns nothing: the -nsb switch of the patched whisper-cli that our script builds, or suppress_blank = false in the whisper.cpp library.
• Decode with beam 5, without timestamps and without carried-over context.
• Run a voice activity detector first and cut long files at pauses into pieces of up to about 25 seconds. The detector can drop faint speech, such as quiet English after Urdu, so turn it off if words go missing (VAD=0 or --no-vad in our scripts).
• In whisper.cpp, q8_0 is the best format on Apple Silicon and q4_0 the fastest on x86 servers.
• On a CPU, use whisper.cpp or faster-whisper: Transformers runs about ten times slower than real time there.

Limitations

Read these before relying on the model.

• Names are its weakest point. The test chapter of our second audiobook has 23 name occurrences: Urdu Sukhan ASR spelled 11 right, the base model 4. Names it rarely heard in training come out as similar common words, for example عمیر (Umair) as امیر (“rich”) and تانیا (Tania) as دانیا. A prompt listing the expected names raised the count from 12 to 17 of 23 in whisper.cpp. FLEURS news, full of foreign names, is where it improves least. When names matter (people, places, brands), give them in a prompt and check them by hand.
• Word spacing and spelling. Words are sometimes joined or split (آنندوہار for آنند وہار), and spelling follows our house style, which can differ from yours.
• No punctuation. Transcripts come without punctuation, like the training text.
• Very faint English after Urdu can come out in Urdu letters or repeat a word.
• Narrator-specific pronunciations, for example a dropped ھ (پھر heard as پر), are sometimes written as pronounced.
• Silence. Without the blank setting above, it writes a word such as اچھا on silence, as every Whisper model does.
• Accents and settings. IndicVoices is mostly Indian Urdu, the Common Voice test has few women, and we have no test of spontaneous phone conversations between Pakistani speakers. Our test sets are small (20 to 300 clips) and we report no confidence intervals.
• Speed on CPUs. Every clip pays for a full 30-second window, so short live clips are slower than real time on small CPUs.
• Not measured. We have not scored or timed Whisper large-v3, measured memory use, or run the model on a phone.

Next version

What we plan to change for the next model, each from a problem this one showed:

Data and credits

Trained on about 110 hours: IndicVoices and the Urdu part of Rasa by AI4Bharat, Mozilla Common Voice, Google FLEURS, the dataset of Muhammad Noman – nzg73 under its own licence, UAT audio only (transcribed by our own model; no public licence; used by agreement with its developers), MUSAN noise and music, and two audiobooks read by two narrators, with transcripts proofread by Asad Shehzad. The research report lists every source and its terms.

Research reportExample audio on Hugging FaceTest sets on Hugging Face

Prefer the browser?

Clean a recording before you transcribe it, free online:

Audio EnhancerEcho RemoverExtract Audio from VideoSilence Remover

Cite

Shehzad, A. (2026). Urdu Sukhan ASR: Accurate Urdu Speech Recognition at Half the Size of Whisper large-v3. Technical report LA-RB-2026-004, Loudness.ai. DOI: 10.5281/zenodo.23518703.

BibTeX and RIS files are on the research report page:

Research reportRead on Zenodo

Links

Hugging FaceGitHubExample audio on Hugging FaceTest sets on Hugging FaceTry it in ColabInstallation guideResearch reportAI Noise Remover SDKloudness-ttsAll developer guides