Learn
What is MMLU-Pro?
Last reviewed July 16, 2026
MMLU-Pro is a broad-knowledge benchmark of 12,000 questions across 14 domains, from law and medicine to math and engineering, with 10 answer choices per question instead of the original MMLU's 4. It was built because MMLU saturated: harder, more reasoning-heavy questions and more choices cut guessing luck from 25% to 10% and restored headroom for distinguishing strong models.
What changed from MMLU
Three moves: questions requiring multi-step reasoning replaced recall-heavy ones, each question offers 10 options so random guessing yields 10%, and noisy or contaminated items were filtered out. The result is a broad test that still separates top models where the original had compressed into its ceiling.
What MMLU-Pro tells you
It measures breadth: consistent performance across 14 unrelated domains signals general knowledge plus applied reasoning rather than a niche specialty. It complements narrow, deep tests (research math, PhD science): a model can lead a specialist benchmark while trailing on breadth, and the reverse.
idapt's reasoning rankings include an MMLU-Pro leaderboard, and every model page lists its available benchmark scores next to per-token prices so breadth and cost read together.
Frequently asked
See it in practice
The fastest way to understand it is to use it. Start free, no credit card.
Try idapt free