Foundation Model for Cardiac Time Series via Masked Latent Attention
Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent foundation models (FMs) have shown promise for learning transferable ECG representations, most existing pretraining approaches treat leads as independent channels and fail to explicitly leverage their strong structural redundancy.
Introduction
In the field of cardiology, the analysis of ECGs is crucial for diagnosing a range of conditions. Traditional methods often overlook the interconnected nature of the leads, which can result in suboptimal performance in predictive tasks. To address this gap, our research introduces a novel approach known as the Latent Attention Masked Autoencoder (LAMAE) foundation model. This model is designed to enhance the understanding of ECG data by utilizing the structural relationships between leads.
Methodology
The LAMAE framework operates by learning cross-lead connection mechanisms during self-supervised pretraining. This innovative approach is pivotal in modeling higher-order interactions across leads. The key features of LAMAE include:
- Latent Attention: This mechanism allows the model to dynamically focus on relevant lead interactions, ensuring that the most informative signals are prioritized.
- Permutation-Invariant Aggregation: The model aggregates information from leads in a manner that is independent of their order, reinforcing the robustness of the learned representations.
- Adaptive Weighting: Each lead’s representation is weighted adaptively based on its contribution to the overall signal, enhancing the model’s ability to discern important features.
Results
We conducted extensive evaluations using the Mimic-IV-ECG database to test the efficacy of our proposed model. The results demonstrated that leveraging the cross-lead connection is an effective form of structural supervision. This approach significantly improved both the quality and transferability of the learned representations.
Specifically, our method outperformed traditional independent-lead masked modeling approaches and alignment-based baselines in predicting ICD-10 codes. The empirical findings suggest that by utilizing the structural redundancy inherent in ECG signals, LAMAE achieves superior performance in tasks that require nuanced understanding of cardiac health.
Conclusion
In summary, the Latent Attention Masked Autoencoder presents a significant advancement in the modeling of ECG data. By explicitly accounting for the structural relationships between leads, LAMAE not only enhances representation quality but also extends the applicability of ECG analysis in clinical settings. Future work will focus on refining this model further and exploring its potential in real-time monitoring and diagnosis of cardiovascular diseases.
References
For more detailed information, please refer to the original paper: arXiv:2603.26475v1.