Readable Minds: Emergent Theory-of-Mind-Like Behavior in LLM Poker Agents
Summary: arXiv:2604.04157v1 Announce Type: new
Abstract: Theory of Mind (ToM) — the ability to model others’ mental states — is fundamental to human social cognition. Whether large language models (LLMs) can develop ToM has been tested exclusively through static vignettes, leaving open whether ToM-like reasoning can emerge through dynamic interaction. Here we report that autonomous LLM agents playing extended sessions of Texas Hold’em poker progressively develop sophisticated opponent models, but only when equipped with persistent memory.
In a 2×2 factorial design crossing memory (present/absent) with domain knowledge (present/absent), each with five replications (N = 20 experiments, ~6,000 agent-hand observations), we find that memory is both necessary and sufficient for ToM-like behavior emergence (Cliff’s delta = 1.0, p = 0.008). Agents with memory reach ToM Level 3-5 (predictive to recursive modeling), while agents without memory remain at Level 0 across all replications.
Key Findings
- Strategic Deception: Strategic deception grounded in opponent models occurs exclusively in memory-equipped conditions (Fisher’s exact p < 0.001).
- Domain Expertise: Domain expertise does not gate ToM-like behavior emergence but enhances its application: agents without poker knowledge develop equivalent ToM levels but less precise deception (p = 0.004).
- Deviation from Optimal Play: Agents with ToM deviate from game-theoretically optimal play (67% vs. 79% TAG adherence, delta = -1.0, p = 0.008) to exploit specific opponents, mirroring expert human play.
- Natural Language Expression: All mental models are expressed in natural language and directly readable, providing a transparent window into AI social cognition.
- Cross-Model Validation: Cross-model validation with GPT-4o yields weighted Cohen’s kappa = 0.81 (almost perfect agreement).
Implications
These findings demonstrate that functional ToM-like behavior can emerge from interaction dynamics alone, without explicit training or prompting. This research has significant implications for understanding artificial social intelligence and biological social cognition. The ability of LLMs to develop sophisticated mental models through gameplay not only enhances their utility in strategic domains but also opens avenues for further research into the cognitive capabilities of AI.
As LLMs continue to evolve, the intersection of memory, dynamic interaction, and social cognition will be critical in shaping their development. Future studies may explore the broader applications of ToM-like behavior in various fields, including social robotics, human-computer interaction, and virtual agents, presenting opportunities for enhanced collaboration between humans and AI systems.