Context-Value-Action Architecture for Value-Driven Large Language Model Agents
Summary: arXiv:2604.05939v1 Announce Type: new
Abstract: Large Language Models (LLMs) have shown promise in simulating human behavior, yet existing agents often exhibit behavioral rigidity, a flaw frequently masked by the self-referential bias of current “LLM-as-a-judge” evaluations. By evaluating against empirical ground truth, we reveal a counter-intuitive phenomenon: increasing the intensity of prompt-driven reasoning does not enhance fidelity but rather exacerbates value polarization, collapsing population diversity.
Introduction
As the field of artificial intelligence continues to evolve, Large Language Models (LLMs) represent a significant advancement in the simulation of human-like behavior. However, these models are not without their limitations. One of the most pressing issues is behavioral rigidity, which can lead to a lack of adaptability in various contexts. This rigidity is often overlooked due to the self-referential evaluations that dominate current assessment methodologies.
Research Findings
Our research indicates that enhancing prompt-driven reasoning does not necessarily improve the fidelity of LLMs. In fact, it can lead to increased value polarization—a phenomenon where differing values become more pronounced, ultimately diminishing the diversity of perspectives represented within model outputs. This finding challenges the prevailing assumption that more intense reasoning correlates with better performance.
Proposed Solution: Context-Value-Action Architecture
To address these challenges, we introduce the Context-Value-Action (CVA) architecture. This innovative framework is grounded in the Stimulus-Organism-Response (S-O-R) model and draws upon Schwartz’s Theory of Basic Human Values. The CVA architecture stands apart from traditional methods by decoupling action generation from cognitive reasoning, allowing for a more nuanced understanding of human values.
Value Verification
A key component of the CVA architecture is the novel Value Verifier. This verifier is trained on authentic human data, enabling it to model dynamic value activation effectively. By doing so, we can ensure that the actions generated by LLMs are not only contextually relevant but also aligned with a broader spectrum of human values.
Experimental Validation
We conducted extensive experiments using CVABench, a benchmark that includes over 1.1 million real-world interaction traces. The results demonstrate that the CVA architecture significantly outperforms existing baselines in several critical areas:
- Mitigation of Polarization: The CVA approach effectively reduces value polarization, fostering a richer diversity of outputs.
- Enhanced Fidelity: By separating reasoning from action, the fidelity of behavior improves, leading to more accurate representations of human-like interactions.
- Increased Interpretability: The CVA architecture offers greater transparency in how values influence decision-making processes in LLMs.
Conclusion
The Context-Value-Action architecture represents a significant step forward in the development of value-driven LLM agents. By addressing the issues of behavioral rigidity and value polarization, this framework not only enhances the fidelity of LLMs but also ensures that they can better reflect the complexity of human values. As we move forward, the implications of this research could pave the way for more adaptable and contextually aware artificial intelligence systems.