Dual-Stage LLM Framework for Scenario-Centric Semantic Interpretation in Driving Assistance
Summary: arXiv:2603.27536v1 Announce Type: new
Abstract
Advanced Driver Assistance Systems (ADAS) increasingly rely on learning-based perception, yet safety-relevant failures often arise without component malfunction, driven instead by partial observability and semantic ambiguity in how risk is interpreted and communicated. This paper presents a scenario-centric framework for reproducible auditing of LLM-based risk reasoning in urban driving contexts.
Introduction
As the automotive industry continues to evolve with the integration of artificial intelligence, the need for robust safety measures in Advanced Driver Assistance Systems (ADAS) becomes increasingly critical. Current systems utilize learning-based perceptions that, while innovative, are not immune to failures. These failures often stem from the complexity of real-world driving environments where semantic ambiguities and partial observability can lead to misinterpretation of risks.
Framework Overview
The proposed dual-stage framework is designed to enhance the reliability of LLM-based reasoning in driving assistance applications. By focusing on scenario-centric audits, the framework establishes a systematic approach to evaluate how risk is interpreted across different models, particularly in urban driving scenarios.
Methodology
The framework involves the construction of deterministic, temporally bounded scenario windows derived from multimodal driving data. These scenario windows are evaluated under fixed prompt constraints and a closed numeric risk schema. Such a structured approach ensures that outputs remain comparable across different models.
Experiments and Results
To validate the effectiveness of the proposed framework, experiments were conducted on a curated set of scenarios where vulnerable road users were present. The study compared two text-only models alongside one multimodal model, all subjected to identical inputs and prompts. The following key findings emerged:
- Systematic inter-model divergence in severity assignment.
- High-risk escalation was observed differently across models.
- Variability in evidence use and causal attribution was significant.
- Disagreement regarding the presence of vulnerable road users was prevalent.
Discussion
The results revealed that the differences in risk assessment often stemmed from intrinsic semantic indeterminacy rather than isolated failures of individual models. This variability emphasizes the necessity for scenario-centric auditing and the management of ambiguities when integrating LLM-based reasoning into safety-critical systems.
Conclusion
In conclusion, the dual-stage LLM framework presents a significant advancement in auditing risk reasoning within ADAS. By addressing semantic ambiguities and promoting structured evaluations, this framework lays the groundwork for improving safety measures in urban driving contexts. Future work should focus on refining the framework and exploring its applications across various driving scenarios to enhance the reliability of automated driving assistance systems.