VLegal-Bench: Cognitively Grounded Benchmark for Vietnamese Legal Reasoning of Large Language Models
Summary: arXiv:2512.14554v5 Announce Type: replace-cross
The rapid advancement of large language models (LLMs) has enabled new possibilities for applying artificial intelligence within the legal domain. However, the complexity, hierarchical organization, and frequent revisions of Vietnamese legislation present significant challenges for evaluating how effectively these models interpret and utilize legal knowledge. To address this gap, the Vietnamese Legal Benchmark (VLegal-Bench) has been introduced as the first comprehensive benchmark designed to systematically assess LLMs on Vietnamese legal tasks.
Introduction to VLegal-Bench
Informed by Bloom’s cognitive taxonomy, VLegal-Bench encompasses multiple levels of legal understanding through tasks designed to reflect practical usage scenarios. The benchmark comprises 10,450 samples generated through a rigorous annotation pipeline, where legal experts label and cross-validate each instance using a dedicated annotation system.
Features of VLegal-Bench
The unique features of VLegal-Bench include:
- Authoritative Grounding: Every sample is grounded in authoritative legal documents, ensuring high-quality data for model evaluation.
- Real-World Workflows: The benchmark mimics real-world legal assistant workflows, including:
- General legal questions and answers
- Retrieval-augmented generation
- Multi-step reasoning
- Scenario-based problem solving tailored to Vietnamese law
- Cognitive Framework: The assessment framework is designed to be standardized, transparent, and cognitively informed, facilitating a robust evaluation of LLM performance.
Significance of VLegal-Bench
By establishing a solid foundation for assessing LLM performance in Vietnamese legal contexts, VLegal-Bench supports the development of more reliable, interpretable, and ethically aligned AI-assisted legal systems. This benchmark is not only pivotal for academic research but also crucial for practitioners seeking to leverage AI within the legal field.
Access and Reproducibility
To facilitate access and reproducibility for researchers and developers, a public landing page for the VLegal-Bench has been created. You can explore the benchmark and its resources at https://vilegalbench.cmcai.vn/.
Conclusion
As the legal field continues to evolve with the integration of AI technologies, benchmarks like VLegal-Bench are essential for ensuring that these advancements are grounded in a solid understanding of the specific legal contexts they are meant to serve. This benchmark represents a significant step forward in the assessment of AI capabilities in the Vietnamese legal landscape.