General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Merging
Summary: arXiv:2604.09609v1 Announce Type: new
Abstract: Human behavior models are essential as behavior references and for simulating human agents in virtual safety assessment of automated vehicles (AVs), yet current models face a trade-off between interpretability and flexibility. General-purpose large language models (LLMs) offer a promising alternative: a single model potentially deployable without parameter fitting across diverse scenarios. However, what LLMs can and cannot capture about human driving behavior remains poorly understood. We address this gap by embedding two general-purpose LLMs (OpenAI o3 and Google Gemini 2.5 Pro) as standalone, closed-loop driver agents in a simplified one-dimensional merging scenario and comparing their behavior against human data using quantitative and qualitative analyses. Both models reproduce human-like intermittent operational control and tactical dependencies on spatial cues. However, neither consistently captures the human response to dynamic velocity cues, and safety performance diverges sharply between models. A systematic prompt ablation study reveals that prompt components act as model-specific inductive biases that do not transfer across LLMs. These findings suggest that general-purpose LLMs could potentially serve as standalone, ready-to-use human behavior models in AV evaluation pipelines, but future research is needed to better understand their failure modes and ensure their validity as models of human driving behavior.
Introduction
The emergence of automated vehicles (AVs) has necessitated the development of robust human behavior models. These models are crucial for simulating human agents during virtual safety assessments, where understanding driver behavior can significantly impact safety outcomes. Traditional models often grapple with a trade-off between being interpretable and flexible, which can limit their effectiveness.
General-purpose LLMs: A New Approach
Recent advancements in artificial intelligence have introduced general-purpose large language models (LLMs) as a potential solution. These models, such as OpenAI’s o3 and Google’s Gemini 2.5 Pro, stand out due to their ability to operate across varied scenarios without the need for extensive parameter adjustments. This adaptability could revolutionize how we test and evaluate human driving behavior in AVs.
Methodology
- Embedding Models: The study involves embedding two LLMs as standalone, closed-loop driver agents.
- Scenario: A simplified one-dimensional merging scenario is utilized to assess the models’ performance.
- Comparative Analysis: The behavior of these models is compared against human driving data using both quantitative and qualitative methods.
Findings
The findings indicate that both LLMs exhibit human-like characteristics in certain aspects of driving behavior:
- They demonstrate intermittent operational control similar to that of human drivers.
- The models display tactical dependencies on spatial cues, a critical aspect of human driving behavior.
However, significant limitations were also identified:
- Neither model consistently captures human responses to dynamic velocity cues, a critical factor in real-world driving scenarios.
- There is a noticeable divergence in safety performance between the two models, raising concerns about their reliability.
Conclusion
The study concludes that while general-purpose LLMs show promise as potential standalone models for simulating human behavior in AV evaluations, more research is required. Understanding their limitations and failure modes is essential to ensure their validity and effectiveness as models of human driving behavior. As the field of automated driving continues to evolve, leveraging these advanced AI models could play a pivotal role in enhancing the safety and reliability of AVs.