Improvisational Games as a Benchmark for Social Intelligence of AI Agents: The Case of Connections
Summary: arXiv:2604.00284v1 Announce Type: new
Abstract: We formally introduce an improvisational wordplay game called Connections to explore reasoning capabilities of AI agents. Playing Connections combines skills in knowledge retrieval, summarization and awareness of cognitive states of other agents. We show how the game serves as a good benchmark for social intelligence abilities of language model based agents that go beyond the agents’ own memory and deductive reasoning and also involve gauging the understanding capabilities of other agents. Finally, we show how through communication with other agents in a constrained environment, AI agents must demonstrate social awareness and intelligence in games involving collaboration.
Introduction to Connections
The game Connections offers a unique framework for evaluating the social intelligence of AI agents. Unlike traditional benchmarks that focus solely on computational ability, Connections requires players to engage in creative wordplay that necessitates social interaction and understanding. This game serves to challenge AI agents not only to recall information but also to interpret and predict the actions of their fellow players.
Core Components of the Game
Connections is structured around several key components that enhance its effectiveness as a benchmark:
- Knowledge Retrieval: AI agents must access a wide range of vocabulary and concepts quickly and efficiently.
- Summarization Skills: Players are required to distill large amounts of information into concise and relevant responses.
- Cognitive Awareness: Understanding the mental states and strategies of other agents is crucial for successful gameplay.
Social Intelligence in AI
Social intelligence is more than just a measure of an agent’s ability to provide correct responses; it entails a deeper understanding of social cues, collaborative strategies, and the dynamics of communication. In the context of Connections, AI agents must adapt to the evolving nature of the game, which requires them to:
- Gauge the intentions of other players.
- Modify their approaches based on the feedback and actions of teammates.
- Communicate effectively to achieve common goals.
Evaluation of AI Agents
The introduction of Connections as a benchmark allows researchers to evaluate AI agents on multiple fronts:
- Performance Metrics: Success in the game can be quantitatively measured, providing a clear picture of an agent’s capabilities.
- Qualitative Analysis: Observations of gameplay can reveal insights into the agents’ reasoning processes and social interactions.
- Adaptability: The ability to adjust strategies based on the behavior of others is a strong indicator of social intelligence.
Conclusion
In conclusion, the game Connections serves as a promising benchmark for assessing the social intelligence of AI agents. By combining knowledge retrieval, summarization, and cognitive awareness, the game pushes the boundaries of what is traditionally expected from AI. Future research should focus on refining these benchmarks and exploring the implications of social intelligence in AI development, ultimately leading to more sophisticated and aware systems.