Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvement
Summary: arXiv:2604.09579v1 Announce Type: new
Abstract: In large-scale cloud service platforms, thousands of customer tickets are generated daily and are typically handled through on-call dialogues. This high volume of on-call interactions imposes a substantial workload on human support analysts. Recent studies have explored reactive agents that leverage large language models as a first line of support to interact with customers directly and resolve issues. However, when issues remain unresolved and are escalated to human support, these agents are typically disengaged. As a result, they cannot assist with follow-up inquiries, track resolution progress, or learn from the cases they fail to address.
In this paper, we introduce Vigil, a novel proactive agent system designed to operate throughout the entire on-call life-cycle. Unlike reactive agents, Vigil focuses on providing assistance during the phase in which human support is already involved. It integrates into the dialogue between the customer and the analyst, proactively offering assistance without explicit user invocation. Moreover, Vigil incorporates a continuous self-improvement mechanism that extracts knowledge from human-resolved cases to autonomously update its capabilities.
Key Features of Vigil
- Proactive Assistance: Vigil engages in conversations even when human agents are involved, offering suggestions and solutions without being explicitly asked.
- Continuous Learning: The system learns from previous cases handled by human agents, enabling it to enhance its capabilities over time.
- Integration: Vigil seamlessly integrates into existing workflows, making it easier for human analysts to leverage its support.
- Real-Time Updates: The agent can provide real-time updates on the status of customer inquiries, improving the overall customer experience.
Deployment and Evaluation
Vigil has been successfully deployed on Volcano Engine, ByteDance’s cloud platform, for over ten months. During this period, comprehensive evaluations have been conducted to assess its effectiveness and practicality. The results have demonstrated significant improvements in response times and resolution rates, alleviating the workload on human support teams.
Impact on Customer Support
The introduction of Vigil is set to transform the customer support landscape by:
- Reducing Response Times: With Vigil’s proactive capabilities, customers receive quicker responses, enhancing satisfaction.
- Improving Issue Resolution: By learning from past interactions, Vigil can suggest solutions that lead to faster issue resolution.
- Enhancing Human-AI Collaboration: The system allows human agents to focus on more complex issues, improving overall efficiency.
Conclusion
Vigil represents a significant advancement in the field of AI-driven customer support. By providing proactive assistance and continuously improving its capabilities, it addresses key challenges faced by customer support teams in large-scale environments. The open-source version of this work is publicly available at https://github.com/volcengine/veaiops, encouraging further research and development in this innovative area.