On the Role of Fault Localization Context for LLM-Based Program Repair
Summary: arXiv:2604.05481v1 Announce Type: cross
Abstract: Fault Localization (FL) is a key component of Large Language Model (LLM)-based Automated Program Repair (APR), yet its impact remains underexplored. In particular, it is unclear how much localization is needed, whether additional context beyond the predicted buggy location is beneficial, and how such context should be retrieved. We conduct a large-scale empirical study on 500 SWE-bench Verified instances using GPT-5-mini, evaluating 61 configurations that vary file-level, element-level, and line-level context.
Our results show that more context does not consistently improve repair performance. File-level localization is the dominant factor, yielding a 15-17x improvement over a no-file baseline. Expanding file context is often associated with improved performance, with successful repairs most commonly observed in configurations with approximately 6-10 relevant files. Element-level context expansion provides conditional gains that depend strongly on the file context quality, while line-level context expansion frequently degrades performance due to noise amplification. LLM-based retrieval generally outperforms structural heuristics while using fewer files and tokens.
Overall, the most effective FL context strategy typically combines a broad semantic understanding at higher abstraction levels with precise line-level localization. These findings challenge our assumption that increasing the localization context uniformly improves APR, and provide practical guidance for designing LLM-based FL strategies.
Key Findings
- File-level localization is a critical factor, resulting in a significant performance boost.
- Expanding file context tends to lead to better repair outcomes, particularly with 6-10 relevant files.
- Element-level context can be beneficial, but its effectiveness is heavily reliant on the quality of file context.
- Line-level context often introduces noise, which can negatively impact repair performance.
- LLM-based retrieval methods are more effective compared to traditional structural heuristics.
Implications for Future Research
This study opens several avenues for future research in the domain of Automated Program Repair. As LLMs continue to evolve, understanding the nuances of fault localization will be crucial. Researchers are encouraged to explore the following:
- Investigating the optimal balance of context types for various programming languages.
- Examining the effects of varying quality in code snippets on repair performance.
- Developing hybrid models that combine the strengths of LLMs with traditional program analysis techniques.
In conclusion, the study emphasizes the importance of context in fault localization and its implications for LLM-based APR. As the field progresses, these insights will be instrumental in refining approaches and enhancing the efficacy of program repair methodologies.