Multi-Agent LLMs for Adaptive Acquisition in Bayesian Optimization
Summary: arXiv:2603.28959v1 Announce Type: cross
Abstract
The exploration-exploitation trade-off is central to sequential decision-making and black-box optimization, yet how Large Language Models (LLMs) reason about and manage this trade-off remains poorly understood. Unlike Bayesian Optimization, where exploration and exploitation are explicitly encoded through acquisition functions, LLM-based optimization relies on implicit, prompt-based reasoning over historical evaluations, making search behavior difficult to analyze or control.
In this work, we present a metric-level study of LLM-mediated search policy learning, studying how LLMs construct and adapt exploration-exploitation strategies under multiple operational definitions of exploration, including informativeness, diversity, and representativeness. We show that single-agent LLM approaches, which jointly perform strategy selection and candidate generation within a single prompt, suffer from cognitive overload, leading to unstable search dynamics and premature convergence.
Introduction
The demand for efficient optimization methods is growing in various fields, including engineering, finance, and data science. Traditional Bayesian Optimization techniques provide a structured approach to navigate the exploration-exploitation dilemma. However, the integration of LLMs introduces novel complexities as they employ implicit reasoning mechanisms.
Challenges of Single-Agent LLM Approaches
Single-agent LLM frameworks attempt to combine strategy selection and candidate generation, but this approach has notable drawbacks:
- Cognitive Overload: The simultaneous processing of multiple functions can overwhelm the model, leading to performance degradation.
- Unstable Search Dynamics: The lack of clarity in decision-making processes results in unpredictable search outcomes.
- Premature Convergence: Single-agent approaches are prone to settling on suboptimal solutions too quickly.
Proposed Multi-Agent Framework
To mitigate the limitations of single-agent systems, we introduce a multi-agent framework that distinctly separates the strategic and tactical components of the optimization process.
- Strategy Agent: This agent is responsible for assigning interpretable weights to various search criteria, making exploration-exploitation decisions explicit.
- Generation Agent: Based on the defined search policy, this agent generates candidates, enabling a more focused search strategy.
Empirical Results
The empirical evaluations conducted on various continuous optimization benchmarks demonstrate the substantial improvements afforded by the multi-agent approach. Key findings include:
- Increased Effectiveness: By decoupling strategic control from candidate generation, search efficiency improves significantly.
- Enhanced Stability: The explicit nature of decision-making minimizes cognitive overload and stabilizes search dynamics.
- Adaptability: The framework allows for easier adjustments to exploration-exploitation strategies based on real-time feedback.
Conclusion
This study highlights the potential of multi-agent LLM frameworks in enhancing Bayesian Optimization methods. By clearly delineating roles within the optimization process, we provide a pathway for more effective and understandable search strategies, paving the way for further advancements in the field.