Enhancing Robustness of Federated Learning via Server Learning
Summary: arXiv:2604.03226v1 Announce Type: cross
Abstract
This paper explores the use of server learning for enhancing the robustness of federated learning against malicious attacks even when clients’ training data are not independent and identically distributed. We propose a heuristic algorithm that uses server learning and client update filtering in combination with geometric median aggregation. We demonstrate via experiments that this approach can achieve significant improvement in model accuracy even when the fraction of malicious clients is high, even more than 50% in some cases, and the dataset utilized by the server is small and could be synthetic with its distribution not necessarily close to that of the clients’ aggregated data.
Introduction
Federated learning is a distributed approach to machine learning that allows multiple clients to collaboratively train a model while keeping their data local. However, this method faces challenges, particularly in terms of robustness against malicious clients who might attempt to compromise the learning process.
Key Concepts
- Federated Learning: A decentralized approach where clients train models locally and only share updates with a central server.
- Server Learning: A technique that utilizes a central server to enhance the learning process by filtering client updates and aggregating them intelligently.
- Geometric Median Aggregation: A method of aggregating multiple data points that minimizes the distance to all points, providing a robust estimate against outliers.
Methodology
The proposed heuristic algorithm integrates server learning with client update filtering. This dual approach allows the server to effectively manage the contributions of clients by identifying and mitigating the effects of malicious updates. The use of geometric median aggregation further enhances the robustness of the model by ensuring that the aggregated updates reflect the majority of the honest clients, rather than being skewed by malicious outliers.
Experimental Results
In extensive experiments, we evaluated the performance of our approach under various conditions, including scenarios where the proportion of malicious clients exceeded 50%. The results indicated a marked improvement in model accuracy compared to traditional federated learning methods. Even with a small and potentially synthetic dataset, our method demonstrated resilience against attacks, illustrating the effectiveness of server learning in real-world applications.
Conclusion
The findings from our research underscore the importance of integrating server learning techniques in federated learning frameworks. By employing client update filtering and geometric median aggregation, we can significantly enhance the robustness of federated learning systems, even in the presence of a high fraction of malicious clients. This advancement not only improves model accuracy but also promotes trust and collaboration in federated environments.
Future Work
Future research will focus on refining the proposed algorithm and exploring its applicability in various domains. Additionally, we aim to investigate the integration of more sophisticated client behavior modeling techniques to further enhance the robustness against adaptive attacks.