Federated learning is a distributed machine learning approach that emphasizes privacy protection. However, it faces two main challenges: privacy leakage and communication overhead. Prior work has addressed these issues individually, focusing either on privacy protection or on reducing communication overhead. While some methods claim to address both issues and offer balanced trade-off solutions, they have been shown to fall short in practice. When being applied to large models, they harms the convergence efficiency and effective. In this paper, we propose a novel approach that effectively balances communication efficiency and security while also achieving superior convergence performance compared to existing algorithms. Our proposed series of quantization methods — Terngrad extension, layer-wise quantization, and twice quantization—are designed to enhance both the stability and efficiency of convergence. Experimental results show that compared to similar algorithm, our method can achieve convergence in larger models, demonstrates robust defense against 8 mainstream attack methods, and can accelerate communication speeds by up to 10 times.
Paper
Full text
Efficient and Secure Federated Learning via Enhanced Quantization and Encryption
OpenAlex · Privacy-Preserving Technologies in Data · 2024
Abstract
Federated learning is a distributed machine learning approach that emphasizes privacy protection. However, it faces two main challenges: privacy leakage and communication overhead. Prior work has addressed these issues individually, focusing either on privacy protection or on reducing communication overhead. While some methods claim to address both issues and offer balanced trade-off solutions, they have been shown to fall short in practice. When being applied to large models, they harms the convergence efficiency and effective. In this paper, we propose a novel approach that effectively balances communication efficiency and security while also achieving superior convergence performance compared to existing algorithms. Our proposed series of quantization methods — Terngrad extension, layer-wise quantization, and twice quantization—are designed to enhance both the stability and efficiency of convergence. Experimental results show that compared to similar algorithm, our method can achieve convergence in larger models, demonstrates robust defense against 8 mainstream attack methods, and can accelerate communication speeds by up to 10 times.