BENCHMARK EXPLORER · reasoning

MMLU-Pro

A more demanding knowledge and reasoning test across academic subjects.

What it measures

Multiple-choice accuracy across the MMLU-Pro evaluation.

Interpretation caveats

Prompting, answer extraction and evaluation versions can change scores.

Original benchmark

Family: reasoning
Published versions: 1
Metric: Accuracy
Evidence: Source-reported

Published results

Version

Models ordered by MMLU-Pro. Click a row to open its model profile.
#			Modalities
1	MiniMaxAI		InputTextOutputText	204,800tokens
2	Qwen		InputTextImageVideoOutputText	262,144tokens
3	moonshotai		InputTextImageOutputText	262,144tokens
4	Qwen		InputTextImageVideoOutputText	262,144tokens
5	Qwen		InputTextImageVideoOutputText	262,144tokens
6	Qwen		InputTextImageVideoOutputText	262,144tokens
7	deepseek-ai		InputTextOutputText	163,840tokens
7	deepseek-ai		InputTextOutputText	163,840tokens
8	moonshotai		InputTextOutputText	262,144tokens
9	Qwen		InputTextOutputText	128,000tokens
9	stepfun-ai		InputTextOutputText	262,144tokens
10	zai-org		InputTextOutputText	204,800tokens
11	deepseek-ai		InputTextOutputText	64,000tokens
12	LGAI-EXAONE		InputTextOutputNot documented	—Not documented
13	nvidia		InputTextOutputNot documented	—Not documented
14	Qwen		InputTextImageVideoOutputText	262,144tokens
15	MiniMaxAI		InputTextOutputText	204,800tokens
16	deepseek-ai		InputTextOutputText	163,840tokens
17	jdopensource		InputTextOutputNot documented	—Not documented
18	Qwen		InputTextOutputText	262,144tokens
19	Qwen		InputTextOutputNot documented	—Not documented
20	nvidia		InputTextOutputText	262,144tokens
21	meituan-longcat		InputTextOutputNot documented	—Not documented
22	nvidia		InputTextOutputNot documented	—Not documented
23	arcee-ai		InputTextOutputNot documented	—Not documented
24	Qwen		InputTextOutputNot documented	—Not documented
25	tiiuae		InputTextOutputNot documented	—Not documented
26	Qwen		InputTextOutputNot documented	—Not documented
27	deepseek-ai		InputTextOutputText	163,840tokens
28	Qwen		InputTextOutputNot documented	—Not documented
29	meta-llama		InputTextOutputText	131,072tokens
30	LiquidAI		InputTextOutputNot documented	—Not documented
31	Qwen		InputTextOutputNot documented	—Not documented
—	openbmb	—	InputTextOutputNot documented	—Not documented
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	Anthropic	—	InputTextImageFileOutputText	1,000,000tokens
—	nvidia	—	InputTextOutputImage	—Not documented
—	nvidia	—	InputTextOutputImage	—Not documented
—	HelpingAI	—	InputTextOutputNot documented	—Not documented
—	zai-org	—	InputTextOutputText	200,000tokens
—	zai-org	—	InputTextOutputText	204,800tokens
—	Google	—	InputTextOutputText	—Not documented
—	Google	—	InputAudioFileImageTextVideoOutputText	1,048,576tokens

Source-backed evidence. Different tasks. A clearer picture.

About Methodology Benchmark dashboard