Title: GSR-GNN: Training Acceleration and Memory-Saving Framework of Deep GNNs on Circuit Graph
Summary: arXiv:2603.27156v1 Announce Type: cross
Abstract
Graph Neural Networks (GNNs) have emerged as a powerful tool for circuit analysis, showcasing significant potential in handling complex circuit graphs. However, the scalability of GNNs to modern large-scale circuit graphs is frequently hindered by GPU memory limitations and high training costs, particularly when deep models are employed. In response to these challenges, we revisit the architecture of deep GNNs specifically for circuit graphs and demonstrate that, when properly trained, these deep models substantially outperform their shallow counterparts. This observation highlights the need for an efficient, domain-specific training framework.
Introduction to GSR-GNN
To address the limitations associated with training deep GNNs, we propose a novel framework called Grouped-Sparse-Reversible GNN (GSR-GNN). This innovative approach facilitates the training of GNNs with architectures featuring up to hundreds of layers while concurrently reducing both computational and memory overhead. The GSR-GNN framework integrates several key components designed to maximize efficiency:
- Reversible Residual Modules: These modules allow for the reconstruction of activations, enabling the reuse of computed values and reducing memory consumption during training.
- Group-Wise Sparse Nonlinear Operator: This operator compresses node embeddings, ensuring that task-relevant information is retained while minimizing memory usage.
- Optimized Execution Pipeline: This pipeline is designed to eliminate fragmented activation storage, which reduces unnecessary data movement and enhances computational efficiency.
Performance and Results
In extensive experiments conducted on sampled circuit graphs, GSR-GNN showcased impressive performance metrics. The framework achieved:
- Up to 87.2% peak memory reduction, making it feasible to work with larger models without exceeding memory limits.
- Over 30 times training speedup, significantly decreasing the time required to train deep GNNs.
- Negligible degradation in correlation-based quality metrics, ensuring that the accuracy and reliability of the models remain intact.
Conclusion
The GSR-GNN framework represents a significant advancement in making deep GNNs practical for large-scale Electronic Design Automation (EDA) workloads. By effectively addressing the memory and training challenges associated with deep learning in circuit analysis, GSR-GNN opens up new possibilities for researchers and industry professionals alike. The ability to train deeper models without compromising on performance or efficiency is a promising step forward in the application of GNNs in complex circuit environments.