High Volatility and Action Bias Distinguish LLMs from Humans in Group Coordination
Summary: arXiv:2604.02578v1 Announce Type: cross
Abstract
Humans exhibit remarkable abilities to coordinate in groups. As large language models (LLMs) become more capable, it remains an open question whether they can demonstrate comparable adaptive coordination and whether they use the same strategies as humans. To investigate this, we compare LLM and human performance on a common-interest game with imperfect monitoring: Group Binary Search. In this n-player game, participants need to coordinate their actions to achieve a common objective. Players independently submit numerical values in an effort to collectively sum to a randomly assigned target number. Without direct communication, they rely on group feedback to iteratively adjust their submissions until they reach the target number.
Key Findings
Our research yields several important insights into the differences between human and LLM coordination in group settings:
- Behavioral Adaptation: Unlike humans, who demonstrate a capacity to adapt and stabilize their behavior over time, LLMs often fail to show improvement across games.
- Excessive Switching: LLMs exhibit excessive switching in their submissions, which negatively impacts group convergence towards the target number.
- Feedback Sensitivity: Humans benefit significantly from richer feedback, such as numerical error magnitude, while LLMs show minimal response to similar enhancements.
Methodology
The study was grounded in a detailed analysis of performance metrics and behavioral dynamics. Key components of our methodology included:
- Reactivity Scaling: This metric assesses how quickly and effectively participants adjust their strategies based on feedback.
- Switching Dynamics: We examined the patterns of submission changes over multiple rounds to identify tendencies toward volatility.
- Learning Across Games: This aspect evaluated whether participants could improve their performance based on previous experiences in the game.
Implications of Findings
These findings suggest that there are fundamental differences in the coordination strategies employed by humans versus LLMs. The inability of LLMs to stabilize their behavior and their tendency to switch actions excessively could hinder their effectiveness in collaborative environments. As AI continues to advance, understanding these discrepancies is crucial for developing more effective collaborative systems that integrate human and machine intelligence.
Future Research Directions
To bridge the coordination gap between LLMs and humans, future research should focus on:
- Enhancing the feedback mechanisms available to LLMs to simulate richer human-like responses.
- Investigating alternative architectures or training methods that could improve LLM adaptability and performance in group settings.
- Exploring the implications of these findings in real-world applications, such as team collaboration tools and AI-assisted decision-making systems.
In conclusion, while LLMs are powerful tools, their current limitations in group coordination highlight the need for continued research and development to achieve human-like adaptive capabilities.