BENCHMARK EXPLORER · math
GSM8K
Mathematical word problems that require several reasoning steps.
What it measures
Answer accuracy on the reported GSM8K evaluation.
Interpretation caveats
Prompting and sample selection can change reported accuracy.
Original benchmarkPublished results
| # | Modalities | Compare | ||||
|---|---|---|---|---|---|---|
| 1 | Qwen | InputTextOutputNot documented | —Not documented | |||
| 2 | deepseek-ai | InputTextOutputText | 163,840tokens | |||
| 3 | microsoft | InputTextOutputNot documented | —Not documented | |||
| 4 | meta-llama | InputTextOutputText | 131,072tokens | |||
| — | openbmb | — | InputTextOutputNot documented | —Not documented | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | Anthropic | — | InputTextImageFileOutputText | 1,000,000tokens | ||
| — | nvidia | — | InputTextOutputImage | —Not documented | ||
| — | nvidia | — | InputTextOutputImage | —Not documented | ||
| — | deepseek-ai | — | InputTextOutputText | 64,000tokens | ||
| — | deepseek-ai | — | InputTextOutputText | 163,840tokens | ||
| — | deepseek-ai | — | InputTextOutputText | 163,840tokens | ||
| — | deepseek-ai | — | InputTextOutputText | 163,840tokens | ||
| — | HelpingAI | — | InputTextOutputNot documented | —Not documented | ||
| — | tiiuae | — | InputTextOutputNot documented | —Not documented | ||
| — | zai-org | — | InputTextOutputText | 204,800tokens | ||
| — | zai-org | — | InputTextOutputText | 200,000tokens | ||
| — | zai-org | — | InputTextOutputText | 204,800tokens | ||
| — | — | InputTextOutputText | —Not documented | |||
| — | — | InputAudioFileImageTextVideoOutputText | 1,048,576tokens | |||
| — | — | InputTextImageVideoFileAudioOutputText | 1,048,576tokens | |||
| — | — | InputTextImageVideoFileAudioOutputText | 1,048,576tokens | |||
| — | — | InputAudioFileImageTextVideoOutputText | 1,048,576tokens | |||
| — | thinky | — | InputTextImageAudioOutputText | 524,288tokens | ||
| — | jdopensource | — | InputTextOutputNot documented | —Not documented | ||
| — | LGAI-EXAONE | — | InputTextOutputNot documented | —Not documented | ||
| — | Not reported | — | InputTextOutputText | —Not documented | ||
| — | Moonshot | — | InputTextImageVideoOutputText | 1,048,576tokens | ||
| — | moonshotai | — | InputTextOutputText | 262,144tokens | ||
| — | moonshotai | — | InputTextOutputText | 262,144tokens | ||
| — | moonshotai | — | InputTextImageOutputText | 262,144tokens | ||
| — | LiquidAI | — | InputTextOutputNot documented | —Not documented | ||
| — | meituan-longcat | — | InputTextOutputNot documented | —Not documented | ||
| — | meituan-longcat | — | InputTextOutputNot documented | —Not documented | ||
| — | XiaomiMiMo | — | InputTextOutputNot documented | —Not documented | ||
| — | MiniMaxAI | — | InputTextOutputText | 204,800tokens | ||
| — | MiniMaxAI | — | InputTextOutputText | 204,800tokens | ||
| — | MiniMaxAI | — | InputTextOutputText | 204,800tokens | ||
| — | miromind-ai | — | InputTextOutputNot documented | —Not documented | ||
| — | miromind-ai | — | InputTextOutputNot documented | —Not documented | ||
| — | Meta | — | InputTextOutputText | —Not documented | ||
| — | Meta | — | InputTextImageVideoFileOutputText | 1,048,576tokens | ||
| — | Meta | — | InputTextImageVideoFileOutputText | 1,048,576tokens |