ChatSVA: Bridging SVA Generation for Hardware Verification via Task-Specific LLMs
Functional verification is a critical component of integrated circuit (IC) development, consuming over 50% of the total lifecycle. Within this domain, SystemVerilog Assertions (SVAs) play a vital role in formal property verification and enhancing simulation-based debugging. However, the manual authoring of SVAs is often labor-intensive and prone to errors, leading to inefficiencies in the verification process.
Recent advancements in artificial intelligence, particularly through Large Language Models (LLMs), have presented a promising opportunity to automate aspects of this process. Nevertheless, the direct application of LLMs in SVA generation has faced significant challenges, primarily due to low functional accuracy and a lack of domain-specific training data.
Introduction to ChatSVA
To tackle these challenges, a new system named ChatSVA has been introduced. This end-to-end SVA generation system is founded on a multi-agent framework designed to enhance the efficiency and accuracy of SVA creation. At the heart of ChatSVA is the AgentBridge platform, which employs a systematic approach to generate high-purity datasets, addressing the data scarcity issues often encountered in few-shot learning scenarios.
Performance Evaluation
ChatSVA was rigorously evaluated on a set of 24 register-transfer level (RTL) designs. The results were impressive:
- Syntax Pass Rate: 98.66%
- Functional Pass Rate: 96.12%
- Average SVAs Generated per Design: 139.5
- Function Coverage: 82.50%
These metrics indicate a significant improvement over previous methods, with a 33.3 percentage point increase in functional correctness and an astonishing 11x enhancement in function coverage compared to the prior state-of-the-art (SOTA) methodologies.
Significance and Future Directions
ChatSVA not only sets a new benchmark in automated SVA generation but also lays the groundwork for effectively addressing long-chain reasoning problems in few-shot, domain-specific scenarios. The introduction of this innovative framework signifies a leap forward in the field of hardware verification, promising to streamline the verification process and reduce the overall time and labor involved.
Furthermore, an online service has been made publicly available, allowing users to experience the capabilities of ChatSVA firsthand. Interested parties can access this service at https://www.nctieda.com/CHATDV.html.
As the demand for faster and more reliable IC development continues to grow, systems like ChatSVA will undoubtedly play an essential role in shaping the future of hardware verification and ensuring the reliability of complex electronic systems.