Improving Role Consistency in Multi-Agent Collaboration via Quantitative Role Clarity
Summary: arXiv:2604.02770v1 Announce Type: new
Abstract
In large language model (LLM)-driven multi-agent systems, disobeying role specifications—defined as the failure to adhere to the responsibilities and constraints of an assigned role—represents a significant failure mode. This discrepancy can result in agents behaving as if they are assigned different roles, which can hinder collaboration and effectiveness. In this paper, we propose a novel approach to enhance role consistency through a method we term quantitative role clarity.
Introduction
The adoption of multi-agent systems powered by large language models has transformed various domains, from customer service to collaborative robotics. However, ensuring that each agent adheres to its designated role remains a challenging problem, often leading to inefficiencies and miscommunication.
Proposed Methodology
To tackle the issue of role inconsistency, we introduce a systematic approach that involves the following key components:
- Role Assignment Matrix: We construct a role assignment matrix S(φ) = [sij(φ)], where sij(φ) measures the semantic similarity between the behavior trajectory of the i-th agent and the role description of the j-th agent.
- Role Clarity Matrix: We define the role clarity matrix M(φ) as softmax(S(φ)) – I, where softmax(S(φ)) is applied row-wise to S(φ) and I is the identity matrix. This formulation allows us to quantify the alignment between the agents’ role descriptions and their behavioral trajectories.
- Regularization During Fine-Tuning: The role clarity matrix M(φ) is then employed as a regularizer during lightweight fine-tuning processes. This integration aims to enhance role consistency and, in turn, improve overall task performance.
Experimental Results
We conducted extensive experiments on the ChatDev multi-agent system to evaluate the effectiveness of our proposed method. Our results indicate a substantial improvement in both role consistency and task performance:
- The role overstepping rate decreased from 46.4% to 8.4% for the Qwen model, and from 43.4% to 0.2% for the Llama model.
- The role clarity score increased significantly, rising from 0.5328 to 0.9097 for Qwen, and from 0.5007 to 0.8530 for Llama.
- Moreover, the task success rate improved from 0.6769 to 0.6909 for Qwen, and from 0.6174 to 0.6763 for Llama.
Conclusion
In conclusion, our approach of implementing quantitative role clarity significantly enhances the consistency of role adherence in multi-agent systems. By integrating a structured methodology that utilizes both role assignment and clarity matrices, we can foster improved collaboration among agents, ultimately leading to higher task success rates.