FLIP:迈向全面且可靠的联邦提示学习评估
FLIP: Towards Comprehensive and Reliable Evaluation of Federated Prompt Learning
摘要 Abstract
随着对隐私和数据安全的日益重视,联邦学习作为一种去中心化的机器学习训练方法得到了广泛应用,无需共享原始数据。提示学习通过微调预训练模型的提示嵌入,在减少计算成本和通信开销的同时,利用视觉-语言模型(如CLIP)的强大性能和泛化能力,在联邦环境中展现出显著优势。本文探讨了联邦学习与提示学习的结合,特别是针对视觉-语言模型的应用。在本研究中,我们提出了一种名为FLIP的综合框架,用于评估联邦提示学习算法。FLIP对8种最先进的联邦提示学习方法在4种联邦学习协议和12个开源数据集上的性能进行了评估,并考虑了6种不同的评估场景。我们的研究表明,提示学习在保持分布内和分布外设置下的强泛化性能的同时,资源消耗极低。这项工作强调了联邦提示学习在数据稀缺、未见类别以及跨域分布偏移环境中的有效性。我们开源了FLIP中实现的所有算法代码,以促进该领域的进一步研究。
The increasing emphasis on privacy and data security has driven the adoption of federated learning, a decentralized approach to train machine learning models without sharing raw data. Prompt learning, which fine-tunes prompt embeddings of pretrained models, offers significant advantages in federated settings by reducing computational costs and communication overheads while leveraging the strong performance and generalization capabilities of vision-language models such as CLIP. This paper addresses the intersection of federated learning and prompt learning, particularly for vision-language models. In this work, we introduce a comprehensive framework, named FLIP, to evaluate federated prompt learning algorithms. FLIP assesses the performance of 8 state-of-the-art federated prompt learning methods across 4 federated learning protocols and 12 open datasets, considering 6 distinct evaluation scenarios. Our findings demonstrate that prompt learning maintains strong generalization performance in both in-distribution and out-of-distribution settings with minimal resource consumption. This work highlights the effectiveness of federated prompt learning in environments characterized by data scarcity, unseen classes, and cross-domain distributional shifts. We open-source the code for all implemented algorithms in FLIP to facilitate further research in this domain.