ELITE: Experiential Learning and Intent-Aware Transfer for Self-improving Embodied Agents
Summary: arXiv:2603.24018v1 Announce Type: new
Abstract: Vision-language models (VLMs) have shown remarkable general capabilities, yet embodied agents built on them fail at complex tasks, often skipping critical steps, proposing invalid actions, and repeating mistakes. These failures arise from a fundamental gap between the static training data of VLMs and the physical interaction for embodied tasks. VLMs can learn rich semantic knowledge from static data but lack the ability to interact with the world. To address this issue, we introduce ELITE, an embodied agent framework with Experiential Learning and Intent-aware Transfer that enables agents to continuously learn from their own environment interaction experiences, and transfer acquired knowledge to procedurally similar tasks.
Mechanisms of ELITE
ELITE operates through two synergistic mechanisms:
- Self-reflective knowledge construction: This mechanism extracts reusable strategies from execution trajectories and maintains an evolving strategy pool through structured refinement operations.
- Intent-aware retrieval: This process identifies relevant strategies from the pool and applies them to current tasks.
Performance Improvements
Experiments conducted on the EB-ALFRED and EB-Habitat benchmarks reveal significant performance enhancements. ELITE achieves a remarkable 9% and 5% performance improvement over base VLMs in the online setting without any supervision. In the supervised setting, ELITE demonstrates its ability to generalize effectively to unseen task categories, outperforming state-of-the-art training-based methods.
Conclusion
The results of these experiments underscore the effectiveness of ELITE in bridging the gap between semantic understanding and reliable action execution. By enabling embodied agents to learn from their interactions and apply this knowledge to new tasks, ELITE represents a significant advancement in the field of artificial intelligence.
Future Implications
The introduction of ELITE opens new avenues for research and development in the realm of embodied agents. As these agents become more adept at learning from their environments and adapting to new challenges, their potential applications could extend across various domains, including robotics, autonomous systems, and interactive AI assistants.