From Theory to Practice: Code Generation Using LLMs for CAPEC and CWE Frameworks
Summary: arXiv:2604.02548v1 Announce Type: cross
Abstract
The increasing complexity and volume of software systems have heightened the importance of identifying and mitigating security vulnerabilities. The existing software vulnerability datasets frequently fall short in providing comprehensive, detailed code snippets explicitly linked to specific vulnerability descriptions, reducing their utility for advanced research and hindering efforts to develop a deeper understanding of security vulnerabilities.
To address this challenge, we present a novel dataset that provides examples of vulnerable code snippets corresponding to Common Attack Pattern Enumerations and Classifications (CAPEC) and Common Weakness Enumeration (CWE) descriptions. By employing the capabilities of Generative Pre-trained Transformer (GPT) models, we have developed a robust methodology for generating these examples.
Methodology
Our approach utilizes GPT-4o, Llama, and Claude models to generate code snippets that exhibit specific vulnerabilities as described in CAPEC and CWE documentation. The methodology encompasses several key steps:
- Model Selection: We selected state-of-the-art LLMs known for their code generation capabilities.
- Data Mapping: Each model was tasked with generating code snippets directly correlated to specific CAPEC and CWE descriptions.
- Validation: Generated code snippets were evaluated for accuracy and relevance to the corresponding vulnerabilities.
Dataset Overview
This dataset not only enhances the understanding of security vulnerabilities in code but also serves as a valuable resource for training machine learning models focused on automatic vulnerability detection and remediation. Preliminary evaluations suggest that the dataset generated by Large Language Models demonstrates high accuracy and can serve as a reliable reference for vulnerability identification systems.
We found consistent results across the three models, with a remarkable 0.98 cosine similarity among the generated code snippets, indicating a high level of agreement in vulnerability representation. The final dataset comprises:
- 615 CAPEC code snippets
- Three programming languages:
- Java
- Python
- JavaScript
This makes it one of the most extensive and diverse resources in this domain.
Conclusion
As the landscape of software development continues to evolve, the need for effective security measures becomes increasingly critical. The dataset we have created not only provides a foundation for understanding the nature of software vulnerabilities but also aids in the development of more sophisticated security tools. The integration of LLMs in generating contextually relevant code snippets marks a significant advancement in the field of cybersecurity, paving the way for future innovations in vulnerability management.