Research · October 2026

Labari Tiny, 22% fewer errors on Beninese French, offline on a phone

About seven hours of recordings made in our studio in Benin were enough to remove more than a fifth of the errors of this 75 MB model, running on a smartphone with no network.

Four billion people that machines barely hear

More than four billion people live in regions whose languages sit at the margins of the corpora used to train speech recognition systems. Wolof, Hausa, Swahili and Darija are hardly rare languages: each has tens of millions of speakers. What they lack is hours of transcribed speech.

National languages are not the only ones affected. French and English as spoken in Cotonou, Lagos or Dakar suffer just as much, with their own prosody and borrowings, and a model trained on the French of France transcribes them poorly.

We measured it on a phone. On everyday sentences read by two Beninese speakers, the official release of Whisper Tiny gets 30.3% of words wrong, close to one word in three. At this error rate, dictation is unreadable and subtitles say something other than what was spoken.

The model has to fit on a phone

In these regions, calling a remote service is not an answer: connectivity is intermittent, mobile data is expensive, and fieldwork happens precisely where the network does not reach. The model has to run on the phone people already carry, with no subscription and no latency.

Whisper Tiny fits that brief: at 75 MB and 37.2 million parameters, the whole speech recognition system fits easily inside a mobile app.

Its weakness is that of every small model. With little capacity, it generalizes poorly beyond what it has seen, and underrepresented varieties are the first thing it stumbles on. That is also what makes it so sensitive to the quality of the data it is trained on.

What seven hours of recordings changed

We produced Beninese French speech data from twenty-three voice-over sessions recorded in our studio with eight native speakers, then trained Whisper Tiny on about seven hours of that corpus. Nothing else changed: we started from the official weights, and the resulting model, Labari Tiny, is exactly the same size as the original.

We then ran both models on the same smartphone, in airplane mode, through our Android test application. It holds the two models and the full test set on the device, and does not even request network access.

Model, on a smartphone, offlineWord error rate
Whisper Tiny, as released by OpenAI30.3%
Labari Tiny, the same model after our seven hours23.5%

The gap is 6.8 points, which means 22.4% fewer errors, measured on the 209 sentences of the test set, none of which was used for training. Nothing changes at inference time, since the model keeps its size.

That leaves the question of leakage, which any technical reader will raise straight away. The test set is a separate public corpus (labari-voice/fr-bj-speech-pilot), recorded independently, and none of its files was used for training. None of its sentences appears in our corpus, none of its clips has an acoustic twin there, and every training run starts again from Whisper's official weights.

It all comes down to the labels

A fine-tune is never better than its labels, and that is where our work lies. Most cheap speech corpora are transcribed by a machine, often by the very model you are trying to improve: you end up teaching it its own mistakes, and the gains level off quickly. Our transcripts come from the script. The voice actor reads a written text, and that text is the reference. Recognition models are only used to find where each sentence was spoken and to cut in the right place; they have no say over the content.

The rest is filtering, automated and unforgiving. A studio session is full of retakes and false starts, sentences read a second time because the first one did not work. Our pipeline aligns the script with the recording, detects these accidents and throws out any clip whose text does not match the audio exactly. Out of twelve hours of processed tracks, we kept eight: a third of the material goes in the bin, on purpose.

Everything we use comes from our studio in Benin. We write the scripts and record the voices ourselves. Nothing is scraped from the web or outsourced: we know who is speaking and under what contract.

What comes next

At 23.5% errors, the model does not understand Beninese French, and we do not claim it does. It makes more than a fifth fewer errors without gaining a single byte, offline, for about seven hours of data. The curve had not flattened when we stopped training, so there is room left.

This result took no new architecture and no compute budget, only properly labeled data. The method works for any underrepresented variety, given a studio and scripts.

Which leaves the question of where the scarcity lies. Compute can be bought and open models downloaded, whereas an hour of transcribed Beninese French still has to be recorded, where its speakers live.

We do not release the training corpus or the model weights, which are reserved for commercial use. The test set is public (huggingface.co/datasets/labari-voice/fr-bj-speech-pilot), and the evaluation method is published at github.com/ahouetonde/whisper-tiny-beninese-french. The repository documents an earlier run on a computer, with a different inference engine, so its figures differ slightly from those measured on the phone.

To build on-device voice models for your devices, or write to sales@labari.dev.