MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis
Summary: arXiv:2604.10126v2 Announce Type: replace-cross
Metamorphic testing (MT) is a widely recognized technique for alleviating the oracle problem in software testing. Despite its advantages, the adoption of MT is often hindered by the challenges associated with constructing effective metamorphic relations (MRs). These challenges frequently stem from the requirement of domain-specific knowledge or the difficulty in obtaining critical insights necessary for MRs.
In this article, we present MR-Coupler, a novel approach designed to leverage functional coupling between methods available in source code to automatically construct MRs and generate metamorphic test cases (MTCs). The MR-Coupler technique identifies pairs of functionally coupled methods, utilizes large language models to generate candidate MTCs, and validates these candidates through test amplification and mutation analysis.
Key Features of MR-Coupler
- Functional Coupling Analysis: MR-Coupler identifies method pairs that exhibit functional coupling, which is a crucial aspect in constructing effective MRs.
- Large Language Models: The integration of large language models aids in the generation of candidate MTCs, enhancing the efficiency and quality of the testing process.
- Validation Mechanism: A novel validation mechanism is employed to reduce false alarms, ensuring that the generated MTCs are both relevant and effective.
- Efficient Enumeration: The approach leverages three functional coupling features to avoid the expensive enumeration of potential method pairs, streamlining the test generation process.
Evaluation and Results
Our evaluation of MR-Coupler was conducted on a dataset consisting of 100 human-written MTCs and 50 real-world bugs. The results demonstrate the efficacy of MR-Coupler in generating valid MTCs:
- MR-Coupler successfully generates valid MTCs for over 90% of tasks.
- The approach improves the generation of valid MTCs by an impressive 64.90% compared to existing baselines.
- False alarms are reduced by 36.56%, showcasing the reliability of the validation mechanism.
- Notably, the MTCs generated by MR-Coupler were able to detect 44% of real bugs, underscoring its practical applicability.
Conclusion
The results of our study highlight the effectiveness of leveraging functional coupling for automated MR construction. MR-Coupler not only addresses the challenges associated with traditional MT but also demonstrates significant improvements in MTC generation and bug detection rates. Furthermore, we have made the MR-Coupler tool and experimental data publicly available to support future research endeavors in this domain.
Overall, MR-Coupler presents a promising advancement in the field of software testing, facilitating the broader adoption of metamorphic testing practices in real-world applications.