When Institutions Deploy AI Without Knowing If It Works for Their Users
Published · July 1, 2026 The Benchmark Score Is Not the Deployment Score Most AI procurement decisions rest on a single question: did the model pass the benchmark? The answer is almost always yes. The model scored well on GLUE, MMLU, or a vendor-supplied accuracy report, and the procurement committee approved the deployment. What the […]
The Benchmark Problem: How Aggregate AI Accuracy Scores Hide the Gaps That Harm African Users
When an AI vendor hands you a benchmark score, they are handing you one number. That number is usually high enough to feel reassuring. It is almost never the number that matters. Our benchmarks show a 29-point accuracy gap — 92% for Standard American English users, 63% for Yoruba-inflected English users — on the same […]
