Automatic Identification of Parallelizable Loops Using Transformer-Based Source Code Representations
Summary: arXiv:2603.30040v1 Announce Type: cross
Abstract
Automatic parallelization remains a challenging problem in software engineering, particularly in identifying code regions where loops can be safely executed in parallel on modern multi-core architectures. Traditional static analysis techniques, such as dependence analysis and polyhedral models, often struggle with irregular or dynamically structured code.
In this work, we propose a Transformer-based approach to classify the parallelization potential of source code, focusing on distinguishing independent (parallelizable) loops from undefined ones. We adopt DistilBERT to process source code sequences using subword tokenization, enabling the model to capture contextual syntactic and semantic patterns without handcrafted features.
Methodology
The approach is evaluated on a balanced dataset combining synthetically generated loops and manually annotated real-world code, using 10-fold cross-validation and multiple performance metrics. The methodology consists of the following key components:
- Data Collection: A balanced dataset is constructed, including both synthetic and real-world code snippets.
- Model Selection: DistilBERT, a lightweight Transformer model, is utilized for processing code sequences.
- Tokenization: Subword tokenization is employed to effectively capture the nuances of programming syntax.
- Evaluation: The model’s performance is assessed using 10-fold cross-validation to ensure robustness and reliability.
Results
Results show consistently high performance, with mean accuracy above 99% and low false positive rates. The findings demonstrate the effectiveness of the proposed approach in accurately identifying parallelizable loops. Key outcomes include:
- Mean accuracy exceeding 99% across various test cases.
- Low false positive rates, enhancing the reliability of the model.
- Improved generalization capabilities compared to prior token-based methods.
- Efficient preprocessing steps, reducing the need for handcrafted features.
Conclusion
The study highlights the potential of lightweight Transformer models for practical identification of parallelization opportunities at the loop level. As software engineering continues to evolve, the ability to automate the parallelization process will be crucial in leveraging modern multi-core architectures effectively. This research not only advances the field of automatic parallelization but also sets the stage for future work in improving software performance through intelligent code analysis.