Efficient cross-silo federated learning using a computing power-aware scheduler

Li, Zilinghan

Permalink

https://hdl.handle.net/2142/124159 Copy

Description

Title

Efficient cross-silo federated learning using a computing power-aware scheduler

Author(s)

Li, Zilinghan

Issue Date

2024-04-15

Director of Research (if dissertation) or Advisor (if thesis)

Kindratenko, Volodymyr

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Federated Learning
• Device Heterogeneity
• Cross-silo Federated Learning

Abstract

Cross-silo federated learning offers a promising solution to collaboratively train robust and generalized machine learning models in domains such as healthcare, finance, and scientific projects lacking a centralized data repository, without transferring datasets and compromising the privacy of sensitive local data. Because of the disparity of computing resources among different federated learning clients (i.e., device heterogeneity), the efficiency of the prevalent synchronous federated learning algorithms is hindered by wasting time in waiting for the slow straggler clients. On the other hand, asynchronous federated learning algorithms face challenges with convergence rate and final model accuracy when dealing with non-identically and independently distributed (non-IID) heterogeneous datasets due to the negative impact of outdated local models and client drifting issues. To tackle these challenges in cross-silo federated learning with heterogeneous clients and data, we introduce FedCompass, an innovative semi-asynchronous federated learning algorithm featuring a server-side computing power-aware scheduler that adaptively assigns varying amounts of training tasks to different clients using the estimation of the computing power of individual clients. FedCompass ensures that multiple locally trained models are received for aggregation almost simultaneously, thereby reducing local model staleness and improving global model performance. Meanwhile, the overall training process remains asynchronous, eliminating delays caused by slow straggler clients. Through experiments with diverse non-IID federated datasets, we demonstrate that FedCompass can achieve faster convergence and higher accuracy compared to other asynchronous algorithms while maintaining superior efficiency over synchronous algorithms in scenarios involving heterogeneous clients. The code for FedCompass is open-sourced at https://github.com/APPFL/FedCompass.

Graduation Semester

2024-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2024 Zilinghan Li

Owning Collections