拜占庭容错联邦多任务表示学习

Byzantine Resilient Federated Multi-Task Representation Learning

摘要 Abstract

本文提出了一种名为BR-MTRL的拜占庭容错多任务表示学习框架,用于处理故障或恶意代理。我们的方法通过共享神经网络模型实现表示学习,其中所有客户端共享固定层,仅保留特定于客户端的最终层。这种结构捕获了客户端之间的共享特征,同时允许个体适应,使其成为在异构联邦设置中利用客户端数据和计算能力以学习个性化模型的一种有前景的方法。为了学习模型,我们采用交替梯度下降策略:每个客户端优化其本地模型,更新最终层,并将共享表示的估计值发送到中央服务器进行聚合。为了防御拜占庭代理,我们采用了几何中值聚合以确保客户端-服务器通信的鲁棒性。我们的方法能够在分布式环境中实现个性化学习的同时保持韧性。我们在基于Amazon Web Services(AWS)平台构建的联邦试验台上实现了所提出的交替梯度下降算法,并将其性能与各种基准算法及其变体进行了比较。通过使用包括CIFAR-10和FEMINIST在内的真实世界数据集进行广泛的实验,我们证明了该方法的有效性和鲁棒性,并展示了其在存在拜占庭对手的情况下对新未见过且数据有限的客户端的可迁移性。

In this paper, we propose BR-MTRL, a Byzantine-resilient multi-task representation learning framework that handles faulty or malicious agents. Our approach leverages representation learning through a shared neural network model, where all clients share fixed layers, except for a client-specific final layer. This structure captures shared features among clients while enabling individual adaptation, making it a promising approach for leveraging client data and computational power in heterogeneous federated settings to learn personalized models. To learn the model, we employ an alternating gradient descent strategy: each client optimizes its local model, updates its final layer, and sends estimates of the shared representation to a central server for aggregation. To defend against Byzantine agents, we employ geometric median aggregation for robust client-server communication. Our method enables personalized learning while maintaining resilience in distributed settings. We implemented the proposed alternating gradient descent algorithm in a federated testbed built using Amazon Web Services (AWS) platform and compared its performance with various benchmark algorithms and their variations. Through extensive experiments using real-world datasets, including CIFAR-10 and FEMINIST, we demonstrated the effectiveness and robustness of our approach and its transferability to new unseen clients with limited data, even in the presence of Byzantine adversaries.