Luwen Technical Report
Summary: arXiv:2604.06737v1
Announce Type: cross
Abstract
Large language models have demonstrated remarkable capabilities across a wide range of natural language processing tasks, yet their application in the legal domain remains challenging due to the specialized terminology, complex reasoning requirements, and rapidly evolving legal knowledge involved. In this paper, we present Luwen, an open-source Chinese legal language model built upon the Baichuan foundation model through three key techniques: continual pre-training on a large-scale legal corpus, supervised fine-tuning with carefully curated legal instruction data, and retrieval-augmented generation integrated with a comprehensive legal knowledge base.
Key Techniques
The development of Luwen is anchored in three primary techniques:
- Continual Pre-training: Utilizing a large-scale legal corpus to enhance the model’s understanding of legal language.
- Supervised Fine-tuning: Employing carefully curated legal instruction data to ensure the model aligns with specific legal tasks.
- Retrieval-Augmented Generation: Integrating a comprehensive legal knowledge base to improve the model’s ability to generate relevant legal information.
Evaluation of Luwen
Luwen has been evaluated on five representative legal tasks, which include:
- Legal Judgment Prediction: Predicting outcomes based on presented legal cases.
- Judicial Examination: Assessing the model’s performance in simulated judicial scenarios.
- Legal Text Summarization: Summarizing lengthy legal documents for easier comprehension.
- Law Article Question Answering: Answering questions based on specific legal articles.
- Judicial Decision Reasoning: Analyzing the reasoning behind judicial decisions.
Experimental Results
The experimental results indicate that Luwen outperforms several strong baselines in each of the evaluated tasks. This performance demonstrates the effectiveness of our approach in adapting general-purpose language models to the legal domain. The findings highlight that Luwen not only enhances understanding of legal terminology but also improves reasoning capabilities, making it a valuable tool for legal professionals, researchers, and educators.
Conclusion
The introduction of Luwen marks a significant step towards bridging the gap between general language models and specialized legal applications. By leveraging advanced techniques such as continual pre-training, supervised fine-tuning, and retrieval-augmented generation, Luwen sets a new standard for legal language models. Future work will focus on expanding the model’s capabilities and further refining its performance across diverse legal contexts.