PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence
In the rapidly evolving field of biosignal intelligence, recent advancements have led to the development of innovative models that leverage multiple physiological signals for improved performance. The latest contribution to this domain is the introduction of PanLUNA, a compact yet powerful multimodal model designed to process and analyze biosignals effectively.
Introduction to PanLUNA
Physiological foundation models (FMs) have demonstrated significant potential in the realm of biosignal representation learning. However, existing models predominantly focus on single modalities such as EEG (electroencephalogram), ECG (electrocardiogram), or PPG (photoplethysmogram). This limitation arises mainly from the scarcity of paired multimodal datasets, restricting the ability to harness the benefits of multimodal analysis.
PanLUNA addresses this challenge by being a 5.4 million parameter pan-modal FM that seamlessly integrates EEG, ECG, and PPG signals within a single shared encoder. This innovative design allows for efficient processing and analysis of multiple biosignals, paving the way for enhanced biosignal intelligence.
Key Features of PanLUNA
PanLUNA introduces several groundbreaking features that set it apart from existing models:
- Channel-Unification Module: Building on the existing framework of LUNA, PanLUNA employs a channel-unification module that treats multimodal channels as entries in a unified query set. This approach is augmented with sensor-type embeddings, facilitating efficient cross-modal early fusion.
- Robustness to Missing Modalities: One of the standout characteristics of PanLUNA is its inherent robustness to missing modalities during inference. This capability is crucial for real-world applications where not all signals may be available.
- Performance Metrics: Despite its compact size, PanLUNA showcases remarkable performance metrics, achieving an 81.21% balanced accuracy on TUAB abnormal EEG detection and a state-of-the-art 0.7416 balanced accuracy on HMC multimodal sleep staging.
- Quantization-Aware Training: The model employs quantization-aware training with INT8 weights, ensuring that it recovers over 96% of full-precision performance. This feature enhances the model’s efficiency and suitability for deployment in low-power environments.
Deployment and Efficiency
PanLUNA’s design is tailored for deployment on ultra-low-power devices. Specifically, it has been tested on the GAP9 RISC-V microcontroller for wearable applications. The performance results are impressive:
- 325.6 ms latency and 18.8 mJ energy consumption per 10-second, 12-lead ECG inference.
- 1.206 seconds latency at 68.65 mJ for multimodal 5-channel sleep staging over 30-second epochs.
Conclusion
In summary, PanLUNA represents a significant advancement in the field of biosignal intelligence by offering a compact, efficient, and robust multimodal model capable of processing EEG, ECG, and PPG signals simultaneously. Its innovative design and exceptional performance metrics position it as a leading solution for future applications in wearable technology and healthcare analytics.