Recent research has shed light on a major hurdle for cutting-edge AI, especially large language models (LLMs), when it comes to generating accurate mathematical proofs. While these AI systems have been celebrated for tackling complex math problems, their real strength is more about crunching numbers than crafting detailed proofs. The study found that not only do LLMs struggle with creating valid proofs, but they also have a habit of insisting their flawed solutions are correct. This tendency to bluff raises questions about their reliability in mathematical reasoning.
The research team, made up of experts, tested several top-notch LLMs using problems from the 2025 USA Mathematical Olympiad (USAMO). Despite the precision that mathematical proofs demand, these AI models often relied on faulty logic and unsupported assumptions, managing an average score of less than 5%.
This study highlights the importance of having humans carefully check AI-generated mathematical results, as these models can’t always be trusted to accurately judge their own correctness. This finding is particularly concerning because AI’s misplaced confidence might lead users to accept incorrect solutions.
Overall, the results point to the need for continuous improvement in AI’s ability to handle complex reasoning tasks, bringing to light the broader issue of trust in AI systems.
