EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis
Summary: arXiv:2601.05808v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) are expected to be trained to act as agents in various real-world environments, but this process relies on rich and varied tool-interaction sandboxes. However, access to real systems is often restricted; LLM-simulated environments are prone to hallucinations and inconsistencies; and manually built sandboxes are hard to scale. In this paper, we propose EnvScaler, an automated framework for scalable tool-interaction environments via programmatic synthesis.
EnvScaler comprises two main components:
- SkelBuilder: This component constructs diverse environment skeletons through a combination of topic mining, logic modeling, and quality evaluation.
- ScenGenerator: This component generates multiple task scenarios and rule-based trajectory validation functions for each environment.
With EnvScaler, we have successfully synthesized 191 unique environments and approximately 7,000 scenarios. These tools have been applied to Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) specifically for the Qwen3 series models. The results from our experiments, evaluated on three distinct benchmarks, demonstrate that EnvScaler significantly enhances the ability of LLMs to effectively solve tasks within complex environments that require multi-turn and multi-tool interactions.
One of the major challenges in training LLMs as agents is the need for varied and robust environments that simulate real-world scenarios. Traditional approaches often rely on limited access to live systems, leading to a reliance on simulated environments. These simulated environments, however, present numerous issues, including hallucinations—where the model generates plausible-sounding but incorrect information—and inconsistencies that can undermine the training process.
EnvScaler addresses these challenges by automating the creation of tool-interaction environments through programmatic synthesis. The SkelBuilder component leverages advanced techniques in topic mining and logic modeling to create a foundation for diverse environments. This ensures that the environments are not only varied but also of high quality, making them suitable for rigorous training of LLMs.
Meanwhile, the ScenGenerator plays a crucial role in enriching these environments by generating numerous task scenarios. By employing rule-based trajectory validation functions, it ensures that the scenarios are consistent and relevant to the tasks at hand. This dual-component approach allows for a scalable solution that can adapt to the evolving needs of LLM training.
The implications of EnvScaler are significant for the future of LLMs. With the capability to quickly generate a wide range of environments and scenarios, researchers and developers can expedite the training process while improving the robustness of the models. This leads to better performance in real-world applications, where LLMs are expected to interact with multiple tools and engage in complex multi-turn dialogues.
In conclusion, the introduction of EnvScaler marks a pivotal advancement in the field of LLM training. By providing a scalable, automated solution for creating diverse tool-interaction environments, it paves the way for more effective training methodologies. For those interested in exploring EnvScaler further, the code and data are publicly available at https://github.com/RUC-NLPIR/EnvScaler.