AI models have recently demonstrated the ability to tackle mathematical problems with the prowess of top-tier mathematicians. However, they still falter on tasks that would seem straightforward for a human.
According to the 2026 AI Index from Stanford, there is a stark contrast in performance among AI systems. For instance, Google’s Gemini Deep Think received a gold medal at the 2025 International Mathematical Olympiad, showcasing its exceptional mathematical capabilities. In contrast, when it comes to reading analog clocks, the leading AI model on a different assessment managed to do so accurately only about 50% of the time.
At the center of the pursuit for artificial general intelligence (AGI) lies this inconsistency. AGI is a theoretical concept referring to an intelligence system that could learn and adapt across various tasks, rather than excelling in a narrow range of problems. An AGI would be expected to handle sophisticated tasks with minimal human input.
The challenge, however, is the lack of consensus among researchers on what constitutes AGI. It is often framed around the idea of “human-level intelligence,” but measuring human intelligence itself presents difficulties, and there is no universally accepted standard for evaluating AGI. Ari Lightman, a digital media professor at Carnegie Mellon University specializing in AI, suggests that a more effective approach would be to evaluate whether a system can learn, innovate, and adjust to new situations. He notes, “We struggle to find common ground on how to evaluate human intelligence, and we are now trying to apply those same imperfect measures to AGI.”
