A beginner’s guide to federated learning
Recently, Google has built one of the most secure and robust cloud infrastructures to process data and improve our services, known as federated learning.
In Federated Learning, a model is formed from the user’s interaction with mobile devices. Federated learning allows mobile phones to collaboratively learn on a shared prediction model while keeping all training data on the device, altering the ability to perform machine learning techniques by the need to store data in the cloud. This method goes beyond using local models that make predictions based on mobile device APIs such as Mobile Vision API or on-device intelligent response, also providing model training to the user. device. A device downloads the current model, improves it by learning from data on the phone it’s in, and then summarizes the changes as a small, targeted update. Only this small update on the model is sent to the cloud, using an encrypted communication method, where it is immediately averaged with other user updates to improve the shared model.
Register for this Session>>
All other training data remains on the affected device and no individual updates are stored in the cloud. Federated learning makes it easy to distribute model training across multiple devices, the technique also helps take advantage of machine learning while minimizing the need and effort to collect user data. Such models allow inference to be made about the device. Today, tech conglomerates are trying to bring their machine learning applications to users’ devices in order to improve privacy and stability at the same time.
How can it be used?
The federated learning model downloads the current model and calculates an updated model on the device itself through the use of advanced computing and in turn using local data. These locally trained models are then sent back from the devices to the central server where they are aggregated i.e. processes like the weight averaging are performed and then a single consolidated and improved overall model is returned to the target devices. Federated learning allows machine learning algorithms to gain experience from a wide variety of datasets, even if they are present in different places.
This approach allows multiple organizations to collaborate on the development of a model, without the need to directly share secure data between them. Over the course of several repetitive training iterations, shared models are exposed to a much larger range of data than they would be on a single organization. Federated learning therefore decentralizes machine learning by eliminating the need to consolidate data in one place. Instead, the same model is trained through multiple iterations in different places. The potential risk of sharing sensitive clinical data with each other is completely eliminated.
The federated learning process
During initial training, the training method returns a trained model to the server. Popular machine learning algorithms such as deep neural networks and support vector machines could be parameters for analysis. Once trained, it encodes with the statistical data models into numerical parameters and they no longer need the training data for inference. So when the device returns the trained model to the server, it does not contain any raw user data. After the server receives the data from its user devices, it updates the base model with the aggregated parameter values of the user-trained models. This federated learning cycle must be repeated several times before the model reaches an optimum level of precision that may be satisfactory enough for the developer and their expectations. Once the model is finally prepared and ready, it can be distributed to all users at once for on-device inference.
Practical paradigms for using federated learning
Several federated learning tasks, such as federated training or assessment with existing machine learning models, can be easily implemented using TensorFlow. Using TensorFlow, federated learning can be implemented even without requiring prior knowledge of how it works under the hood, and also offers components for evaluating federated learning algorithms implemented on a variety of models. and existing data.
The interfaces offered consist of the following three key elements:
- Models: Classes and helper functions that allow you to encapsulate existing models for use with TFF. Encapsulation of a model can be done by calling a single encapsulation function, i.e. tff.learning.from_keras_model, or by defining a subclass of the tff.learning.Model interface for a full customization.
- Federated calculation generators: These are helper functions that help build federated calculations for learning or assessment, using existing models.
- Datasets: Data collections that you can download and access in Python for use in simulating federated learning scenarios. While federated learning is designed to be used with decentralized data that cannot simply be uploaded to a centralized location, to conduct initial experiments for research purposes, data that can be uploaded and manipulated locally, which becomes especially useful for developers. Two examples of tests with datasets are image classification and text generation.
Example: visualization of MNIST figures for a particular customer
import collections import numpy as np import tensorflow as tf import tensorflow_federated as tff
Image code and source
Example 2: Creating a list of datasets from a given set of users
def make_federated_data(client_data, client_ids): return [ preprocess(client_data.create_tf_dataset_for_client(x)) for x in client_ids ]
Tensorflow also offers two levels of aggregation process for federated learning:
- Local aggregation: it is the aggregation of several batches of examples belonging to an individual customer.
- Federated aggregation: This is the aggregation between several clients (devices) of the system.
Disadvantages of federated learning
Federated learning currently cannot solve all machine learning problems, for example learning to recognize different breeds of dogs by training on carefully labeled examples. If the model becomes gigantic to run on the end user’s device, a developer may need to find other ways to maintain user privacy. When training data is available on the user’s device, sometimes data engineers may encounter difficulties by not having a way to evaluate the data and ensure that it will be of full benefit to the patient. the application. For this reason, federated learning should currently be limited to only those applications where user data does not require any preprocessing. Federated learning currently appears to be better suited for unsupervised learning applications such as language modeling, among others. But Google continues to advance its state of the art for cloud-based ML and research to broaden the range of issues that can be solved using federated learning. They’re also working on improving the language models that power keyboards based on what you’re actually typing on your phone and provide photo ratings based on the types of photos people are viewing, sharing, or deleting.
Federated learning appears to have created a new era of safe and secure AI and appears to have many potentials to work on in the future. It provides a method to secure sensitive user information, but also aggregates the results and identifies common patterns of a complete set, making the pattern robust. He can train according to his user data while keeping it safe. Training and testing with Federated Learning seems to be getting smarter and more self-reliant. As it is still in its early stages and currently faces many challenges with its design and deployment, it is too early to deliver a perfect verdict.
In this article, we understood what federated learning is and how it works. We also discussed how it can be used and the process under the hood. You can try Federated Learning by accessing a sample notebook provided by TensorFlow using the link here.
Easy to understand federated learning
TF website for federated learning
Subscribe to our newsletter
Receive the latest updates and relevant offers by sharing your email.
Join our Telegram Group. Be part of an engaging community