Neighbourhood Transformer: Switchable Attention for Monophily-Aware Graph Learning
Summary: arXiv:2604.08980v1 Announce Type: cross
Abstract
Graph neural networks (GNNs) have become a cornerstone in numerous engineering applications, from social network analysis to chemical research and computer vision. Despite their growing popularity, the efficacy of GNNs is often undermined by the inherent homophily assumption. This assumption posits that connected nodes in a graph tend to share similar attributes, a notion that fails in the context of heterophilic graphs, where connections frequently exist between dissimilar nodes. To address this critical limitation in graph learning, we introduce the Neighbourhood Transformers (NT), a novel framework that leverages self-attention mechanisms within each local neighbourhood, as opposed to the traditional approach of aggregating messages centered on a single node.
Introduction
The Neighbourhood Transformer is designed to be inherently monophily-aware. This design choice ensures that NT has an expressiveness that is theoretically on par with conventional message-passing frameworks. By focusing on local neighbourhoods rather than a central node, the framework captures the complexities inherent in heterophilic graphs.
Key Features of Neighbourhood Transformers
- Self-Attention Mechanism: NT employs self-attention within local neighbourhoods, allowing for more nuanced message processing.
- Monophily Awareness: The design inherently accounts for monophily, improving performance on diverse graph types.
- Neighbourhood Partitioning Strategy: This strategy, combined with switchable attentions, reduces space consumption by over 95% and time consumption by up to 92.67%, making NT feasible for larger graphs.
- Extensive Experimentation: NT has been rigorously tested across 10 real-world datasets, including both heterophilic and homophilic graphs.
Performance and Results
The results from extensive experiments demonstrate that the Neighbourhood Transformer outperforms existing state-of-the-art methods in node classification tasks. This achievement highlights its superior performance and adaptability across various domains. The experiments included five heterophilic and five homophilic datasets, showcasing NT’s robust capability to generalize across different types of graphs.
Conclusion
The Neighbourhood Transformer represents a significant advancement in graph learning methodologies, addressing the limitations posed by homophily assumptions in traditional GNNs. With its innovative approach to message processing and its efficiency in handling larger graphs, NT is poised to enhance the effectiveness of graph-based applications across multiple fields.
Availability
To facilitate reproducibility and encourage industrial adoption, the full implementation code for Neighbourhood Transformers is publicly available at https://github.com/cf020031308/MoNT.