Experience Transfer for Multimodal LLM Agents in Minecraft Game
Summary: arXiv:2604.05533v1 Announce Type: new
Abstract
Multimodal LLM agents operating in complex game environments must continually reuse past experience to solve new tasks efficiently. In this work, we propose Echo, a transfer-oriented memory framework that enables agents to derive actionable knowledge from prior interactions rather than treating memory as a passive repository of static records.
Key Features of Echo
To make transfer explicit, Echo decomposes reusable knowledge into five dimensions:
- Structure: Understanding the framework of tasks.
- Attribute: Identifying the characteristics of different tasks.
- Process: Recognizing the sequence of actions taken in similar tasks.
- Function: Determining the purpose and outcome of actions.
- Interaction: Examining how tasks relate to each other in various contexts.
Methodology
This formulation allows the agent to identify recurring patterns shared across different tasks and infer what prior experience remains applicable in new situations. Building on this formulation, Echo leverages In-Context Analogy Learning (ICAL) to retrieve relevant experiences and adapt them to unseen tasks through contextual examples.
Experimental Results
Experiments conducted in the Minecraft environment demonstrate the effectiveness of Echo. Under a from-scratch learning setting, Echo achieves a remarkable speed-up of 1.3x to 1.7x on object-unlocking tasks. This signifies a substantial improvement in the efficiency of the agents in handling complex tasks.
Moreover, Echo exhibits a burst-like chain-unlocking phenomenon, rapidly unlocking multiple similar items within a short time interval after acquiring transferable experience. This rapid capability suggests that the framework not only enhances speed but also improves the overall adaptability of agents in interactive environments.
Conclusion
The results indicate that experience transfer is a promising direction for improving the efficiency and adaptability of multimodal LLM agents in complex interactive environments. By rethinking how memory is utilized, Echo sets a new standard for the development of intelligent agents capable of navigating and thriving in dynamic settings.