Event-Adaptive State Transition and Gated Fusion for RGB-Event Object Tracking
Summary: arXiv:2604.13426v1 Announce Type: cross
Abstract: Existing Vision Mamba-based RGB-Event (RGBE) tracking methods suffer from using static state transition matrices, which fail to adapt to variations in event sparsity. This rigidity leads to imbalanced modeling—underfitting sparse event streams and overfitting dense ones—thus degrading cross-modal fusion robustness. To address these limitations, we propose MambaTrack, a multimodal and efficient tracking framework built upon a Dynamic State Space Model (DSSM). Our contributions are twofold.
- Event-Adaptive State Transition Mechanism: We introduce a dynamic state transition mechanism that modulates the state transition matrix based on the density of the event stream. A learnable scalar governs the state evolution rate, enabling differentiated modeling of both sparse and dense event flows.
- Gated Projection Fusion Module: We develop a Gated Projection Fusion (GPF) module for robust cross-modal integration. This module projects RGB features into the event feature space and generates adaptive gates from event density and RGB confidence scores. These gates precisely control the fusion intensity, suppressing noise while preserving complementary information.
Experiments demonstrate that MambaTrack achieves state-of-the-art performance on the FE108 and FELT datasets, showcasing its capability to handle diverse tracking scenarios effectively. The framework’s lightweight design suggests promising potential for real-time embedded deployment in various applications.
Introduction
Recent advancements in object tracking have highlighted the limitations of traditional methods, particularly in the context of RGB-Event (RGBE) tracking. Most existing approaches rely on static state transition matrices, which fail to account for the dynamic nature of event data. This oversight often leads to suboptimal performance, especially when dealing with varying event densities.
Methodology
MambaTrack leverages a Dynamic State Space Model (DSSM) to enhance tracking accuracy and efficiency. By incorporating an event-adaptive state transition mechanism, the framework can adjust to different densities of event streams in real time. This adaptability is crucial for achieving balanced performance in tracking, as it allows the model to underfit or overfit according to the specific conditions of the input data.
Key Contributions
- Dynamic Adaptation: The event-adaptive state transition mechanism is designed to respond to real-time variations in event density, which optimizes the tracking process.
- Enhanced Fusion: The GPF module ensures that the integration of RGB and event data is both effective and efficient, enhancing the robustness of the tracking system.
Results
The experimental results indicate that MambaTrack outperforms existing RGBE tracking methods on benchmark datasets, including FE108 and FELT. The framework’s ability to maintain high accuracy while operating in real-time scenarios underscores its viability for deployment in practical applications.
Conclusion
In summary, MambaTrack represents a significant advancement in RGBE tracking methodologies. By addressing the limitations of static state transition matrices through innovative mechanisms for dynamic adaptation and robust fusion, this framework sets a new standard for effective object tracking in diverse environments.