OpenAI starts creating new benchmarks that more accurately evaluate AI models across different languages and cultures

OpenAI is developing new AI benchmarks that better evaluate how models understand different languages and cultures, addressing shortcomings in existing tests that focus mainly on English and translation tasks. The company launched IndQA, covering 12 Indian languages and 10 cultural domains with 2,278 questions created by 261 local experts including journalists and scholars. This benchmark tests real cultural understanding rather than just language translation, with plans to expand similar region-specific evaluations worldwide.