Run Urdu Sukhan ASR

1Choose a format

• whisper.cpp (q8_0, q5_0 or q4_0): the command line and apps. q8_0 (874 MB) is the best choice on Apple Silicon and q4_0 (474 MB) the fastest on x86 servers.
• faster-whisper (CTranslate2 int8): Python on a CPU, the fastest way to transcribe long recordings there.
• Transformers (fp16, about 1.6 GB): Python with a graphics card, and further training. On a CPU alone it is about ten times slower than real time, so use one of the other two formats there.

All three formats give the same kind of results; the research report lists the scores of each runtime. The scripts were tested on macOS and Ubuntu 24.04 with Python 3.12.

2Request access to the model

The model is free for non-commercial use under the CC BY-NC 4.0 licence, and its files are shared on request:

• Sign in to Hugging Face and open the model page.
• Fill in the short form (name, organisation, country and intended use) and accept the terms. We approve each request by hand.
• Once your request is approved, create a read token at huggingface.co/settings/tokens.

Until then, every download in this guide fails with a 401 error. For commercial use, contact us.

3Optional: try it in Colab first

Once your request is approved, you can try the model without installing anything in our Colab notebook. It runs in your own Google Colab session, on a free GPU or the CPU: record from the microphone, upload a file or try the example clips, with the recommended settings built in. It needs a free Google account and your Hugging Face read token. Your audio goes to your Colab session on Google’s servers, never to Loudness.ai.

4Get the scripts

The model’s GitHub repository holds a tested script for each format in its examples folder. Download it and create a Python 3.12 virtual environment inside it. Use Python 3.12, the version the scripts were tested with: some of the packages have no ready-made builds for newer versions yet. On macOS, brew install python@3.12 adds it.

git clone https://github.com/Loudness-ai/urdu-sukhan-asr.git
cd urdu-sukhan-asr
python3.12 -m venv .venv
source .venv/bin/activate

Install the Hugging Face command line tool and log in with the read token from step 2; the scripts download the model with it:

python -m pip install huggingface_hub==2.2.0
hf auth login

The scripts read WAV, FLAC, MP3 and OGG files. Convert other formats first, for example with FFmpeg:

ffmpeg -i recording.m4a -ar 16000 -ac 1 speech.wav

5whisper.cpp: transcribe with the script

You need git, CMake and a C++ compiler. Then transcribe a file:

bash examples/whisper_cpp.sh speech.wav

The first run builds whisper.cpp v1.9.5 with a small patch that adds the -nsb switch, so that silence gives no text, and downloads the q8_0 model and the Silero voice detector into ~/urdu-sukhan-asr. Later runs start straight away. On an x86 server, choose the q4_0 file and the number of threads:

QUANT=q4_0 THREADS=8 bash examples/whisper_cpp.sh interview.mp3

The script runs the Silero voice detector first, which can drop faint speech such as quiet English after Urdu. VAD=0 turns it off, and silence still gives no text:

VAD=0 bash examples/whisper_cpp.sh urdu-then-english.wav

6whisper.cpp: the settings

From ~/urdu-sukhan-asr, the script runs this command, which you can reuse in your own scripts:

whisper.cpp/build/bin/whisper-cli -m models/ggml/ggml-urdu-sukhan-asr-v1-q8_0.bin -l ur -t 8 -bs 5 -nt -mc 0 -nsb \
  --vad -vm whisper.cpp/models/ggml-silero-v5.1.2.bin -vsd 2000 -vp 300 -f speech.wav

• -l ur sets the language (en for English); auto runs the encoder twice for each window.
• -bs 5 uses beam search, -nt leaves out timestamps and -mc 0 stops earlier text from carrying over.
• -nsb lets the model return nothing, so silence, noise and music give no text. In an app that uses the whisper.cpp library, set suppress_blank = false instead.
• --vad skips the parts without speech, and -vsd 2000 -vp 300 keep short pauses inside speech. The voice detector can also drop faint speech, such as quiet English after Urdu: if words are missing, run the script with VAD=0, or this command without --vad -vm … -vsd 2000 -vp 300. -nsb still gives no text on silence.

7Or use faster-whisper in Python

Install faster-whisper and the libraries the script uses, in the versions it was tested with:

python -m pip install faster-whisper==1.2.1 ctranslate2==4.8.3 huggingface_hub==2.2.0 soundfile==0.14.0 soxr==1.1.0

Then transcribe. The first run downloads only the CTranslate2 folder of the model:

python examples/transcribe_faster_whisper.py recording.wav
python examples/transcribe_faster_whisper.py lecture.flac --batch-size 8 --timestamps
python examples/transcribe_faster_whisper.py urdu-then-english.wav --no-vad

The Silero voice detector cuts the audio into speech pieces, which are transcribed in batches with beam 5 and without carried-over text. --no-vad turns the detector off for faint speech such as quiet English after Urdu, and silence still gives no text. The script uses int8 on a CPU and float16 on an NVIDIA graphics card; --threads sets the CPU threads (8 by default).

8Or use Transformers in Python

Install the tested versions of everything the scripts use, including Transformers and its torch dependency:

python -m pip install -r examples/requirements.txt

Then transcribe. The first run downloads about 1.6 GB:

python examples/transcribe_transformers.py speech.wav
python examples/transcribe_transformers.py interview.flac --beams 5 --language ur

The script uses an NVIDIA graphics card or the graphics processor of a Mac with Apple Silicon. Recordings longer than 30 seconds are cut into 30-second pieces and transcribed in batches. --beams 1 (the default) is the fastest; --beams 5 is slightly more accurate. On a CPU alone it is about ten times slower than real time: use whisper.cpp or faster-whisper there.