CURE: Circuit-Aware Unlearning for LLM-based Recommendation
Summary: arXiv:2604.04982v1
Announce Type: cross
Abstract
Recent advances in large language models (LLMs) have opened new opportunities for recommender systems by enabling rich semantic understanding and reasoning about user interests and item attributes. However, as privacy regulations tighten, incorporating user data into LLM-based recommendation (LLMRec) introduces significant privacy risks, making unlearning algorithms increasingly crucial for practical deployment.
Despite growing interest in LLMRec unlearning, most existing approaches formulate unlearning as a weighted combination of forgetting and retaining objectives while updating model parameters in a uniform manner. Such formulations inevitably induce gradient conflicts between the two objectives, leading to unstable optimization and resulting in either ineffective unlearning or severe degradation of model utility. Moreover, the unlearning procedure remains largely black-box, undermining its transparency and trustworthiness.
Introduction to CURE
To tackle these challenges, we propose CURE, a circuit-aware unlearning framework that disentangles model components into functionally distinct subsets and selectively updates them. Here, a circuit refers to a computational subgraph that is causally responsible for task-specific behaviors.
Key Features of CURE
The CURE framework is designed to enhance the unlearning process in LLM-based recommendation systems by focusing on the following key features:
- Core Circuit Extraction: We extract the core circuits underlying item recommendation and analyze how individual modules within these circuits contribute to the forget and retain objectives.
- Categorization of Modules: Based on our analysis, the modules are categorized into forget-specific, retain-specific, and task-shared groups. This categorization is crucial for applying function-specific update rules.
- Mitigation of Gradient Conflicts: By applying distinct update rules to different groups of modules, we effectively mitigate the gradient conflicts that typically arise during the unlearning process.
Results and Experiments
Experiments on real-world datasets have shown that our approach achieves more effective unlearning compared to existing baselines. The results indicate that circuit-aware strategies significantly enhance the stability and utility of LLM-based recommender systems, allowing for a more transparent and trustworthy unlearning process.
Conclusion
In conclusion, CURE represents a significant advancement in the field of LLM-based recommendations by addressing critical challenges related to unlearning and privacy. The circuit-aware approach not only improves the efficacy of unlearning but also reinforces the trustworthiness of recommender systems in an era where data privacy is paramount.