Have you ever noticed how AI can whip up an essay or create stunning art, yet it falters when it comes to reading a clock or figuring out dates on a calendar? A recent study from the University of Edinburgh shines a light on this curious gap in AI’s capabilities. It turns out that even the most advanced AI systems stumble over tasks most of us mastered in elementary school.
Researchers found that these cutting-edge models have a tough time interpreting clock hands and answering questions about dates. While recognizing shapes is a breeze for AI, understanding the nuances of analog clocks and calendars is a different ball game. It requires spatial awareness, context, and basic math—areas where AI still struggles.
Why does this matter? Well, if we can overcome these hurdles, AI could become a real game-changer in time-sensitive applications like scheduling assistants or tools for those who are visually impaired. The study tested multimodal large language models (MLLMs) on various clock styles, including those with Roman numerals and different dials. The results were telling: AI got the clock-hand positions right less than 25% of the time. It had particular trouble with Roman numerals and stylized hands, and even removing the second hand didn’t help much. Clearly, there’s a persistent issue with how AI detects hands and interprets angles.
When it came to calendar questions, like identifying holidays or calculating dates, the AI models didn’t fare much better. They had a 20% error rate, even among the top performers. These findings will be shared at the upcoming Reasoning and Planning for Large Language Models workshop during the Thirteenth International Conference on Learning Representations (ICLR) in Singapore.
Rohit Saxena, who led the study at the University of Edinburgh’s School of Informatics, put it well: “Most people can tell the time and use calendars from an early age. Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people. These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation, and assistive technologies.”
Aryo Gema, also from the School of Informatics, added, “AI research today often emphasizes complex reasoning tasks, but ironically, many systems still struggle when it comes to simpler, everyday tasks. Our findings suggest it’s high time we addressed these fundamental gaps. Otherwise, integrating AI into real-world, time-sensitive applications might remain stuck at the eleventh hour.”
It’s fascinating and a bit ironic, isn’t it? As we push the boundaries of what AI can do, let’s not forget to ensure it can also handle the basics. After all, the real world requires a blend of both complex and simple skills.
