🔍 Read the full analysis: What Egyptian And Nigerian English Bring To The Open ASR Leaderboard on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face has added two private Liva AI speech-evaluation splits to the Open ASR Leaderboard: 1.34 hours of Egyptian English and 1.55 hours of Nigerian English. The additions broaden the accents evaluated, but the short datasets and limited information about their speakers mean results should not be treated as representative of either country’s English speakers overall.
Hugging Face has added two private English speech evaluation splits from Liva AI to its Open ASR Leaderboard, covering Egyptian and Nigerian accents, as described in the original analysis. The 2.89 hours of conversational recordings give users new ways to inspect speech-recognition performance across accents, though the short, private datasets do not establish how systems perform for speakers across either country as a whole.
The new splits are named Liva AI Conversational EG and Liva AI Conversational NG. Hugging Face says the Egyptian set contains 1.34 hours of audio and the Nigerian set contains 1.55 hours. Both consist of spontaneous conversation. Their transcripts are punctuated and cased, and retain disfluencies, according to the leaderboard announcement.
The splits are private evaluation data, not publicly available benchmark datasets. Hugging Face says private test material is intended to make evaluations more trustworthy by reducing the opportunity for developers to tune systems directly to examples available in public test sets. The announcement does not report how the new splits affect model scores or existing leaderboard rankings.
The leaderboard’s Private tab will show two comparisons: US versus non-US English, and Inner Circle versus Outer and Expanding Circles. The non-US group includes the UK, Australia, Canada, India, Nigeria and Egypt. The second grouping draws on Kachru’s Three Circles framework; in the current presentation, the Inner Circle includes the US, UK, Australia and Canada, while India, Nigeria and Egypt are grouped as Outer and Expanding Circles.
Egyptian and Nigerian English join the Open ASR Leaderboard
Hugging Face added private Liva AI speech splits to broaden accent evaluation. The short recordings add coverage, while leaving questions about how representative the results are.
What was added
Both datasets contain spontaneous conversation with punctuated, cased transcripts that retain disfluencies.
Liva AI Conversational EG
A private evaluation split for Egyptian-accented English.
Liva AI Conversational NG
A private evaluation split for Nigerian-accented English.
New ways to compare accents
The leaderboard’s Private tab will show results across two groupings.
- Non-US includes the UK, Australia, Canada, India, Nigeria and Egypt.
- Inner Circle compared with Outer and Expanding Circles, based on Kachru’s framework.
- Hugging Face plans to separate the latter circles as more accents are added.
Why private evaluation matters
Keeping test examples private may make it harder to tune systems directly to known benchmark material.
Users can inspect recognition results for two additional English varieties.
Scores describe these specific short test sets, not all speakers in either country.
More Accents in ASR Comparisons
Speech recognition systems can perform differently across accents. Adding Egyptian and Nigerian English gives leaderboard users a way to examine results for two varieties not included in its previously listed accent groupings. That can make an aggregate score less likely to stand in for performance across all speakers without showing how results vary by accent.
The change is a limited expansion, not a complete measure of recognition quality for either country. The two splits contain less than three hours of speech combined, and the announcement does not provide an independent assessment of their representativeness. Scores will reflect performance on these particular test sets; they should not be read as proof of how a model performs for every Egyptian or Nigerian English speaker.
Private evaluation data can make direct tuning to the test examples harder, which is the rationale Hugging Face gives for using private splits. However, the announcement offers no comparative results showing whether the change alters model rankings or how much it improves the leaderboard’s coverage in practice.
Products Worth Considering
Sony WH-1000XM5 Premium Noise Cancelling Wireless Headphones, Midnight Blue
Bose QuietComfort Wireless Noise-Cancelling Headphones, Cypress Green
Bose QuietComfort 45 SE Wireless Noise-Cancelling Over-Ear Headphones
As an affiliate, we earn on qualifying purchases.
Earlier Private Benchmark Additions
The Liva AI additions continue the leaderboard’s use of private evaluation data. Hugging Face says it previously added 11 private splits from Appen and DataOceanAI, covering scripted and conversational speech across five accents. It later added a private split for Indian-accented English in collaboration with Voice Arena.
The new accent groupings also organize results by broader categories. Hugging Face credits Hume AI with the leaderboard idea and says the comparisons draw on Kachru’s Three Circles framework. The company plans to separate the Outer and Expanding Circles into distinct groups as more accents are added, but has not announced when that change will happen.
“We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian and Nigerian.”
— Hugging Face
What the Short Splits Represent
The announcement does not say when the recordings were collected, how many speakers took part, or how they were selected. It gives no breakdown by region, age or other speaker characteristics, leaving readers without information needed to judge how well the recordings reflect the range of Egyptian and Nigerian English.
Hugging Face has not published baseline scores or explained the effect of the additions on existing rankings in the announcement. It also does not say whether the audio or detailed dataset documentation will become public; the splits are described as private. The recordings’ small total duration and the missing speaker details limit what can be inferred from results.
More Accents and Separate Groups
Hugging Face says it plans to continue adding accents and languages and to split the current Outer and Expanding Circles grouping once more accents are available. No date or threshold for that change was given. For now, users can find the new accent comparisons on the leaderboard’s Private tab.
Further information on speaker composition, collection methods and per-accent results would help readers interpret the scores and assess the reach of the new evaluation coverage. The announcement does not identify a date for additional data or results.
Key Questions
Which accents were added to the Open ASR Leaderboard?
The new private Liva AI splits cover Egyptian-accented English and Nigerian-accented English.
How much speech is in the two splits?
The Egyptian split contains 1.34 hours and the Nigerian split 1.55 hours, for a combined total of 2.89 hours.
What kind of recordings do the splits contain?
Hugging Face describes both as spontaneous conversational audio. The transcripts are punctuated and cased, and include disfluencies.
Can the public download the evaluation data?
No public access is described. Hugging Face identifies the splits as private evaluation data; the announcement does not say whether the recordings or detailed documentation will later be released.
Do these results represent all Egyptian and Nigerian English speakers?
The announcement does not provide enough information to make that judgment. It gives no speaker count, selection method or demographic breakdown, and the recordings total less than three hours. Scores should be understood as results on these specific test sets.
Primary source: Hugging Face · via ThorstenMeyerAI.com
