Are LLMs Smarter Than Chimpanzees? An Evaluation on Perspective Taking and Knowledge State Estimation
Summary: arXiv:2601.12410v2 Announce Type: replace
Abstract: Cognitive anthropology suggests that the distinction of human intelligence lies in the ability to infer other individuals’ knowledge states and understand their intentions. In comparison, our closest animal relative, chimpanzees, lack the capacity to do so. With this paper, we aim to evaluate LLM performance in estimating other individuals’ knowledge states and their potential actions. We design two tasks to test:
- Whether LLMs can predict story characters’ next actions based on their own knowledge vs. improperly using information unavailable from their perspective.
- Whether LLMs can detect when story characters, through their actions, demonstrate knowledge they should not possess.
Recent advancements in large language models (LLMs) have sparked debates about their cognitive capabilities relative to human intelligence and animal cognition. This study delves into two key aspects: perspective taking and knowledge state estimation, drawing comparisons between LLMs and chimpanzees, who are often regarded as benchmarks for animal intelligence.
The first task presented in the study evaluates the ability of LLMs to predict the actions of story characters. This involves analyzing whether LLMs can accurately deduce what a character might do next based solely on the character’s available knowledge. The challenge lies in ensuring that the model does not rely on information that the character would not logically possess. This task is crucial as it reflects a fundamental aspect of human reasoning—the ability to separate one’s own knowledge from that of others.
The second task involves assessing whether LLMs can recognize actions that imply knowledge beyond what a character should have. This ability is essential for understanding intentions and motivations, a skill that distinguishes humans from many other species. In this context, the study highlights that chimpanzees, despite their cognitive limitations, may still outperform LLMs in specific scenarios that require understanding of knowledge states.
Results from the study reveal a concerning trend: most current state-of-the-art LLMs perform at near-random levels on both tasks. When compared to human performance, LLMs fall significantly short, underscoring a critical gap in their cognitive abilities. The findings suggest that while LLMs excel in generating human-like text, they struggle with tasks that require deep understanding of others’ mental states.
The implications of these findings are profound. They call into question the extent to which LLMs can be considered “intelligent” in a human-like sense. The authors argue that future research should prioritize enhancing LLMs’ abilities in knowledge estimation and intention understanding, as these are crucial for advancing artificial intelligence beyond its current limitations.
In conclusion, while LLMs demonstrate remarkable capabilities in language generation, the evaluation of their cognitive skills reveals significant deficiencies when compared to both human intelligence and the cognitive abilities of chimpanzees. This study serves as a stepping stone for further exploration into the nuanced realms of artificial intelligence and its potential to mimic human-like understanding.