A Study on the Impact of Fault Localization Granularity for Repository-Scale Code Repair Tasks
In the realm of software engineering, automatic program repair is a complex challenge, particularly when addressing intricate issues at a repository level. These tasks often encompass multiple stages, including issue reproduction, fault localization, code repair, testing, and validation. Such issues frequently arise in popular GitHub repositories and datasets derived from them.
Introduction
Many existing repository-level approaches divide the process of fault localization and repair into distinct phases. Within these frameworks, the granularity of fault localization varies significantly. While some studies have examined the impact of granularity on smaller datasets, they often conflate this concern with the separate issue of localization accuracy, typically testing code repairs under the assumption of perfect fault localization.
To the best of the authors’ knowledge, there has been no comprehensive repository-scale study that explicitly investigates the effects of granularity under these assumptions. Moreover, previous research has not systematically compared different granularity levels in isolation, which is critical for a nuanced understanding of this issue.
Methodology
To address this gap, we propose a novel framework designed to perform rigorous testing by modifying the localization phase of the Agentless framework. This modification enables the retrieval of ground-truth localization data, which is then integrated as context in the prompt provided to the repair phase. By utilizing this framework, we aim to elucidate the relationship between fault localization granularity and subsequent code repair efficacy.
Findings
Our research presents preliminary findings based on a generalization over the SWE-Bench-Mini dataset. The results indicate that function-level granularity yields the highest repair rates when compared against line-level and file-level granularity. However, a deeper exploration reveals that the ideal granularity for fault localization may indeed be dependent on the specific task at hand.
Conclusion
This study is not aimed at surpassing the current state-of-the-art methodologies in automatic program repair, nor do we intend for our results to be benchmarked against any comprehensive agentic frameworks. Instead, we present our findings as a proof of concept that highlights the potential influence of fault localization granularity on automatic code repair in repository-scale scenarios.
We believe that these preliminary findings will serve as a catalyst for further research into the intricate relationship between fault localization and code repair processes. Future investigations may uncover new insights that could ultimately enhance the effectiveness of automatic program repair techniques.
Call to Action
Researchers and practitioners in the field of software engineering are encouraged to explore this relationship further, as understanding the nuances of fault localization granularity could lead to significant improvements in automated code repair methodologies.