Semi-Supervised Federated Adaptive Label Learning with Disambiguation Prototype

Abstract

Traditional federated learning depends on having data that is categorized or labeled for its training processes. However, obtaining such specific labeled data can be a complex task due to concerns about privacy, the expensive nature of labeling, or a shortage of specialized knowledge for categorizing data in certain fields. To overcome these obstacles, researchers have introduced Federated Semi-Supervised Learning (FSSL). This approach merges a limited quantity of categorized data with a vast amount of uncategorized data, thereby improving the effectiveness of models while maintaining the privacy and security of the data. Despite its advantages, FSSL encounters challenges such as gradual progress in learning and diminished precision, especially in situations where the distribution of data categories is uneven.

To mitigate these problems, we have introduced an innovative framework named Federated Adaptive Label Learning (FedALL). FedALL integrates strategies from transfer learning, partial label learning, and prototype learning. It constructs an Adaptive Label List through transfer learning and devises a Label Disambiguation Prototype via representation learning. These strategies significantly reduce the adverse impacts caused by incorrect one-hot-label predictions for unlabeled data on the client's model. Additionally, incorporating pre-trained weights substantially accelerates the convergence speed of FedALL.

In the experimental section of our paper, we compared FedALL against other baseline models in both Independent and Identically Distributed (IID) and Non-Independent and Identically Distributed (Non-IID) settings across three distinct datasets. The results demonstrate that FedALL outperforms all baselines in all scenarios, especially in Non-IID environments. Moreover, we utilized the Banach Fixed Point Theorem to prove the convergence of the Label Disambiguation Prototype.