基于参数冻结与功率控制的双时间尺度无线联邦学习方法
A Two-Timescale Approach for Wireless Federated Learning with Parameter Freezing and Power Control
摘要 Abstract
联邦学习(FL)允许分布式设备在保护数据隐私的同时协作训练共享的机器学习(ML)模型。然而,资源受限的移动设备面临模型参数密集的计算和通信开销问题。本文观察到,在训练过程中,模型参数往往在收敛之前就趋于稳定。基于这一观察,我们提出了一种通过联合优化冻结稳定参数和控制不稳定参数的发送功率的双时间尺度FL框架,以平衡能耗与收敛性能。首先,我们分析了模型参数冻结和不可靠传输对收敛率的影响。接着,我们构建了一个以最小化模型收敛误差为目标,同时满足能量预算约束的参数冻结比例和发送功率的双时间尺度优化问题。为了解决该问题,我们将其分解为并行子问题,并利用Lyapunov优化方法将每个子问题进一步分解为两个不同时间尺度的问题。在线推导出最优的参数冻结和功率控制策略。实验结果表明,所提出的方案相较于基准方案具有优越性。
Federated learning (FL) enables distributed devices to train a shared machine learning (ML) model collaboratively while protecting their data privacy. However, the resource-limited mobile devices suffer from intensive computation-and-communication costs of model parameters. In this paper, we observe the phenomenon that the model parameters tend to be stabilized long before convergence during training process. Based on this observation, we propose a two-timescale FL framework by joint optimization of freezing stabilized parameters and controlling transmit power for the unstable parameters to balance the energy consumption and convergence. First, we analyze the impact of model parameter freezing and unreliable transmission on the convergence rate. Next, we formulate a two-timescale optimization problem of parameter freezing percentage and transmit power to minimize the model convergence error subject to the energy budget. To solve this problem, we decompose it into parallel sub-problems and decompose each sub-problem into two different timescales problems using the Lyapunov optimization method. The optimal parameter freezing and power control strategies are derived in an online fashion. Experimental results demonstrate the superiority of the proposed scheme compared with the benchmark schemes.