Illocutionary Explanation Planning for Source-Faithful Explanations in Retrieval-Augmented Language Models
Summary: arXiv:2604.06211v1 Announce Type: cross
Abstract: Natural language explanations produced by large language models (LLMs) are often persuasive, but not necessarily scrutable: users cannot easily verify whether the claims in an explanation are supported by evidence. In XAI, this motivates a focus on faithfulness and traceability, i.e., the extent to which an explanation’s claims can be grounded in, and traced back to, an explicit source.
Introduction
In recent years, large language models (LLMs) have gained significant attention for their ability to generate coherent and contextually relevant explanations. However, a critical concern has emerged regarding the scrutability of these explanations. Users often grapple with validating the claims made in the outputs of LLMs, leading to questions about the reliability of information provided.
Research Focus
This study investigates the principles of faithfulness and traceability within the context of retrieval-augmented generation (RAG) models, particularly in the realm of programming education. Textbooks serve as authoritative sources of evidence, making them an ideal benchmark for evaluating the performance of different LLMs.
Methodology
The research benchmarks six LLMs using 90 Stack Overflow questions, which are grounded in three distinct programming textbooks. A quantitative assessment of source faithfulness is conducted through source adherence metrics. The findings reveal stark contrasts between RAG and non-RAG models:
- Non-RAG models displayed a median source adherence of 0%.
- Baseline RAG systems exhibited low median adherence ranging from 22% to 40%, depending on the model.
Illocutionary Theory Application
Inspired by Achinstein’s illocutionary theory of explanation, the study proposes illocutionary macro-planning as a guiding design principle for enhancing source-faithful explanations. This concept is instantiated through a novel approach known as chain-of-illocution prompting (CoI), which expands an initial query into implicit explanatory questions that facilitate more effective retrieval of relevant information.
Results
The implementation of CoI across various models resulted in statistically significant improvements in source adherence, with gains of up to 63%. Nonetheless, it is important to note that absolute adherence levels remain moderate, and the enhancements are not uniformly significant across all models.
User Study
A user study involving 165 retained participants (from an initial recruitment of 220) was conducted to assess the impact of CoI on user satisfaction, relevance, and perceived correctness of the explanations. The results indicated that the improvements in source adherence did not detrimentally affect user satisfaction or the perceived quality of the explanations.
Conclusion
This research highlights the importance of developing source-faithful explanations in LLMs, particularly in educational contexts. The application of illocutionary macro-planning through CoI showcases a promising direction for enhancing the reliability of AI-generated explanations while maintaining user satisfaction.