CORTEG: Foundation Models Enable Cross-Modality Representation Transfer from Scalp to Intracranial Brain Recordings
In a groundbreaking development in brain-computer interface (BCI) technology, researchers have introduced CORTEG, a novel cross-modality transfer framework that leverages large pretrained scalp-EEG foundation models (EEG FMs) to enhance the decoding of intracranial electrocorticography (ECoG) data. This advancement addresses the challenges posed by the limited per-patient data typically available for ECoG, allowing for more effective cross-patient learning.
Understanding the Challenges in ECoG
Intracranial ECoG is known for its high-signal-to-noise ratio, providing valuable access to cortical activity. However, the scarcity of data from individual patients has often led previous approaches to develop small, subject-specific decoders. Such methods frequently overlook the shared information across multiple patients, resulting in less robust performance. CORTEG aims to bridge this gap by utilizing the extensive knowledge embedded in scalp-EEG models.
Key Features of CORTEG
CORTEG incorporates several innovative components that facilitate effective adaptation from scalp EEG to ECoG:
- Pretrained EEG FM Backbone: Utilizing a robust foundation model trained on extensive scalp EEG data provides a strong starting point for ECoG decoding.
- Electrode-Aware KNNSoftFourier Spatial Adapter: This component ensures that the spatial characteristics of the electrodes are taken into account during the transfer process, enhancing the model’s efficacy.
- Dual-Stream Tokenizer: By processing low-frequency and high-gamma activity streams separately, the model captures a more comprehensive representation of brain signals.
- Leave-One-Subject-Out Fine-Tuning Strategy: This approach allows the model to adapt quickly to unseen patients in a matter of 10-30 minutes on a single GPU, making it highly efficient.
Evaluation and Results
The effectiveness of CORTEG was tested on two challenging regression tasks: public finger trajectory regression involving nine participants and private audio envelope regression with sixteen participants. The results demonstrated that CORTEG not only matched but, in many cases, exceeded the performance of traditional task-specific baselines.
- On the public finger trajectory task, CORTEG achieved the highest mean correlation among the methods compared, although the gain was not statistically significant across the nine subjects.
- For the private audio envelope task, CORTEG exhibited larger and statistically significant improvements, showcasing its ability to perform well even with limited data per patient.
Implications for Future Research
The findings from CORTEG provide compelling evidence that scalp-EEG pretraining can be effectively repurposed for ECoG decoding. This capability opens up new avenues for developing data-efficient intracranial BCIs that can adapt to new patients with minimal calibration time. Moreover, the feature analyses performed align with established neurophysiological principles, suggesting that the latent manifolds captured by the model accurately reflect low-dimensional structures related to finger movements.
As the field of brain-computer interfaces continues to evolve, CORTEG represents a significant step forward in leveraging existing scalp EEG data to enhance the decoding of intracranial signals, potentially paving the way for more effective and accessible BCI technologies in clinical settings.
Related AI Insights
- Dynamic Tiered AgentRunner for Governable Enterprise AI
- E-TCAV: Efficient Concept-Based Neural Network Interpretability
- EmbodiSkill: Adaptive Skill Evolution for Embodied Agents
- Safety Risks of Malicious Knowledge Editing in AI Models
- AutoScout24 Boosts Engineering with AI Workflows
- Efficient Active Testing of Large Language Models
- Prospective Compression in Human Abstraction Learning Explained
- TimeClaw: Advanced AI for Time-Series Exploratory Learning
- Hypothesis-Driven Deep Research with Large Language Models
- Automated Algorithm for Infinite-State Polynomial Reachability Games
