KUET at StanceNakba Shared Task: StanceMoE – Mixture-of-Experts Architecture for Stance Detection
In a groundbreaking advancement in the field of natural language processing, researchers from Khulna University of Engineering & Technology (KUET) have presented their innovative model, StanceMoE, at the StanceNakba Shared Task. This model introduces a Mixture-of-Experts (MoE) architecture specifically tailored for actor-level stance detection, a crucial aspect of understanding the nuanced positions authors express towards geopolitical actors in their texts.
The research, detailed in their recent paper available on arXiv (arXiv:2604.00878v1), addresses the limitations of existing transformer-based models that, while effective, often rely on a unified representation that may not effectively capture the diverse linguistic cues essential for accurate stance classification. The complexity of human language, with its contrastive discourse structures, framing cues, and significant lexical indicators, necessitates a more adaptive approach to stance detection.
Abstract
Actor-level stance detection aims to determine an author’s expressed position toward specific geopolitical actors mentioned or implicated in a text. Although transformer-based models have achieved relatively good performance in stance classification, they typically rely on unified representations that may not sufficiently capture heterogeneous linguistic signals, such as contrastive discourse structures, framing cues, and salient lexical indicators. This motivates the need for adaptive architectures that explicitly model diverse stance-expressive patterns. In this paper, we propose StanceMoE, a context-enhanced Mixture-of-Experts (MoE) architecture built upon a fine-tuned BERT encoder for actor-level stance detection. Our model integrates six expert modules designed to capture complementary linguistic signals, including global semantic orientation, salient lexical cues, clause-level focus, phrase-level patterns, framing indicators, and contrast-driven discourse shifts. A context-aware gating mechanism dynamically weights expert contributions, enabling adaptive routing based on input characteristics. Experiments are conducted on the StanceNakba 2026 Subtask A dataset, comprising 1,401 annotated English texts where the target actor is implicit in the text. StanceMoE achieves a macro-F1 score of 94.26%, outperforming traditional baselines, and alternative BERT-based variants.
Key Features of StanceMoE
The StanceMoE model stands out due to its unique design and functionality:
- Mixture-of-Experts Architecture: This innovative structure allows the model to use multiple expert modules, each specializing in different aspects of linguistic expression.
- Context-Aware Gating Mechanism: The model dynamically adjusts the contributions of each expert based on the input, enhancing its adaptability and accuracy.
- Fine-Tuned BERT Encoder: Leveraging the power of BERT, StanceMoE builds a robust foundation for understanding complex language patterns.
- High Performance: With a macro-F1 score of 94.26%, StanceMoE has demonstrated superior performance compared to traditional models and other BERT variants.
Conclusion
The introduction of StanceMoE marks a significant step forward in the field of stance detection, providing researchers and practitioners with a powerful tool for analyzing the subtleties of language in relation to geopolitical discourse. As the need for accurate and nuanced stance detection grows, models like StanceMoE will play a vital role in advancing our understanding of text and its implications.