Analysis and Implementation of Different Open-Source Federated Learning Frameworks to Assess their Technical Implications

Abstract

Google introduced Federated Learning, an approach to decentralized machine learning model training, in 2016. It is designed to allow the use of private data to train machine learning models without the need to possess the data or even "see" it. The main premise of Federated Learning is a paradigm shift from the traditional centralized machine learning training workflow to a distributed setting. In this setting, users carry out the training locally without ever revealing their data and only share the results of their efforts anonymously as model parameter updates, either to a local server or a network of other users. Over the years, several Federated Learning frameworks have emerged, each offering different sets of settings and serving either a broad or a particular purpose. While several comparisons have been made to determine the framework with the most comprehensive set of features, no comparison is available to assess their utility and the implications of using them at an empirical level. This case study uses the popular frameworks NVFlare, Flower, and Federated Scope to evaluate and showcase their main strengths and potential drawbacks, emphasizing the use of an external dataset and model. The results showed that regardless of whether the frameworks displayed considerable strengths in certain areas, there is still room for improvement, and that, even if they simplify the implementation of Federated Learning, a factor of manual work still needs to be taken into account, regardless of the framework at hand. Ultimately, the frameworks have relevant features and areas of opportunity that anyone looking to adopt Federated Learning will need to consider; however, the technical analysis should give a broad perspective on the implications of using the chosen frameworks.