Combee: Scaling Prompt Learning for Self-Improving Language Model Agents
Summary: arXiv:2604.04247v1 Announce Type: new
Abstract: Recent advances in prompt learning allow large language model agents to acquire task-relevant knowledge from inference-time context without parameter changes. For example, existing methods (like ACE or GEPA) can learn system prompts to improve accuracy based on previous agent runs. However, these methods primarily focus on single-agent or low-parallelism settings. This fundamentally limits their ability to efficiently learn from a large set of collected agentic traces.
It would be efficient and beneficial to run prompt learning in parallel to accommodate the growing trend of learning from many agentic traces or parallel agent executions. Yet without a principled strategy for scaling, current methods suffer from quality degradation with high parallelism.
Introduction to Combee
To improve both the efficiency and quality of prompt learning, we propose Combee, a novel framework to scale parallel prompt learning for self-improving agents. Combee speeds up learning and enables running many agents in parallel while learning from their aggregate traces without quality degradation.
Key Features of Combee
- Parallel Scans: Combee leverages parallel scans to enhance the efficiency of the learning process.
- Augmented Shuffle Mechanism: This mechanism ensures that the learning from various agentic traces is well-integrated and maximized.
- Dynamic Batch Size Controller: A unique feature that balances between learning quality and processing delay, ensuring optimal performance.
Performance Evaluation
Combee has been evaluated on multiple benchmarks, including:
- AppWorld: A complex environment for testing language model agents.
- Terminal-Bench: A benchmark designed for evaluating command-line interface interactions.
- Formula: A test suite focusing on mathematical problem-solving capabilities.
- FiNER: A framework for Named Entity Recognition tasks.
Results from these evaluations demonstrate that Combee achieves up to 17x speedup over previous methods while maintaining comparable or improved accuracy at equivalent costs. This significant advancement indicates a promising direction for future research and development in scaling prompt learning methodologies.
Conclusion
The introduction of Combee marks a substantial step forward in the field of self-improving language model agents. By efficiently managing parallel prompt learning, this framework not only addresses the limitations of existing methods but also sets a new standard for performance in the domain. As the need for more advanced AI capabilities grows, innovations like Combee will be crucial in enabling the evolution of intelligent agents capable of learning from extensive datasets.