GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models
The recent advancements in artificial intelligence have brought significant attention to the development and deployment of large language models (LLMs). However, the operational costs associated with serving these models remain a considerable challenge. As highlighted in the latest research paper on arXiv (2604.19398v1), a novel structured pruning framework called GRASPrune has emerged, designed to tackle these issues effectively.
LLMs are known for their high memory and latency costs, primarily due to the extensive number of model parameters, attention computations, and key-value (KV) caches they utilize. The introduction of GRASPrune aims to alleviate these burdens by implementing a strategic approach to model pruning.
Understanding GRASPrune
GRASPrune employs a structured pruning methodology that is applied post-pretraining. This framework simultaneously prunes feedforward neural network (FFN) channels and KV head groups while adhering to a single global budget. Unlike traditional methods that learn importance scores without any constraints, GRASPrune adopts a more efficient approach.
- Lightweight Gate Scores: Instead of applying the budget only after training, GRASPrune learns lightweight gate scores using a projected straight-through estimator. This innovative technique ensures a hard mask that meets budget requirements at every training step while keeping the backbone weights frozen.
- Calibration of Scaling Factors: After establishing the mask, the framework calibrates scaling factors on the retained units to address potential scale mismatches that may arise from the pruning process. These factors are folded into the pruned weights, resulting in a smaller, dense checkpoint that requires no additional parameters during inference.
Performance Insights
The efficacy of GRASPrune has been demonstrated through its application on the LLaMA-2-7B model. The results are impressive, with GRASPrune successfully removing 50% of the parameters from the model. Despite this significant reduction in size, the model achieves a perplexity score of 12.18 on the WikiText-2 dataset.
Furthermore, GRASPrune maintains competitive average zero-shot accuracy across five benchmark tests. This achievement is particularly notable considering that the model was calibrated using only four epochs on 512 unlabeled calibration sequences, utilizing a single NVIDIA A100 80GB GPU, and did not undergo any full model fine-tuning.
Conclusion
GRASPrune represents a significant step forward in the optimization of large language models, offering a structured and efficient pruning methodology that addresses the high costs of model deployment. By leveraging lightweight gating and careful calibration, GRASPrune not only reduces the model’s size but also preserves its performance, making it a promising solution for the future of AI applications.