As it is evident from the title itself, this article talks about an AI framework called Flower used for Federated Learning’s use cases. But before going into the details of the framework, let us have a quick introduction of the Federated Learning approach itself.

What is Federated Learning ?

Conventional Machine Learning models require the training data to be accumulated at a particular data center or on a specific machine. Suppose a Machine Learning project needs data to be collected from say a hundred different sources. Collecting all the data, sending it to a centralized device or data center for computations and storing it on the cloud may become a tedious task depending upon the complexity of  data. Federated Learning approach eliminates the need to centralize the training data and allows ML models to be trained using data residing on mobile devices.

How does it work?

Several mobile devices can download the ML model to be used for the project. Each of the devices can then use its own stored data for improvising the model. Only the model updates and not the entire training data is  sent to the cloud. At the central device or the data center, updates received in an encrypted form from all the users are aggregated for improving the shared model. The updated version of the model is then sent back to the participating devices from the centralized server.

Advantages of Federated Learning approach

• Training data need not be uploaded and stored at a central site.
• As data stays with the clients, time lag gets reduced as transmission of data between server and clients does not occur.
• Data privacy can be maintained since data remains at the local devices.
• No individual update of the model from different clients needs to be stored centrally.
• The prediction process can continue even when there is a lack of Internet connection since the models reside on the local devices only.

For detailed information of the approach, refer to the following sources:

• https://ai.googleblog.com/2017/04/federated-learning-collaborative.html
• https://federated.withgoogle.com
• https://analyticsindiamag.com/top-technologies-to-achieve-security-and-privacy-of-sensitive-data-in-ai-models
• https://analyticsindiamag.com/federated-learning
• https://analyticsindiamag.com/coding-tools-federated-learning

Now that we have covered a brief description of the Federated Learning approach, let’s move on to the topic of main focus.

Flower framework

Flower is a user-friendly framework designed for implementing the Federated Learning approach. 

Installation

Installing the Flower framework requires Python 3.6 or higher version.

To install its stabte version found on PyPI:

pip install flwr

To install its latest (though unstable) releases:

pip install flwr-nightly

To install its latest version from GitHub
pip install git+https://github.com/adap/flower.git

Practical implementation 

Here, we implement the Federated Learning approach using Flower and TensorFlow. Apart from Flower, install TensorFlow using the command:

pip install tensorflow

Flower Client

Now, we create a client file say cli.py following the steps given below:

1. Import TensorFlow and Flower frameworks first.

   import tensorflow as tf
   import flwr as flower 

2. Load the CIFAR10 image classification dataset using Keras utilities of TensorFlow. Detailed description of the dataset can be found here.

   

(x_train, y_train), (x_test, y_test) =  tf.keras.datasets.cifar10.load_data()

     Visit this page to see how tf.keras.datasets.cifar10.load_data()works.

3. We are using the MobileNetV2 neural network architecture here. A comprehensive description of the architecture can also be found in this article.

     Instantiate the model using tf.keras.applications.MobileNetV2()

     

model = tf.keras.applications.MobileNetV2((32, 32, 3), classes=10, weights=None)

4. Compile the model using Adam optimizer and sparse categorical cross entropy as loss function. 

     

model.compile("adam", "sparse_categorical_crossentropy", metrics=["accuracy"])

5. The Flower server selects a Flower client and sends some instructions to it about how to train the model. On getting those instructions, the concerned client calls one of the methods of ‘Client’ – an interface that the server uses to interact with the clients. 

     When we use Keras, Flower framework provides us with a class named ‘NumPyClient’  for implementing the ‘Client’ interface. 

     Implement the three methods of the interface as follows:

            class Cifar_client(flower.client.NumPyClient):
                   def get_parameters(self):
                         return model.get_weights()
                  def fit(self, parameters, config):
                        model.set_weights(parameters)
                        model.fit(x_train, y_train, epochs=1, batch_size=32, steps_per_epoch=5)
                        return model.get_weights(), len(x_train)
                def evaluate(self, parameters, config):
                      model.set_weights(parameters)
                      loss, accuracy = model.evaluate(x_test, y_test)
          return len(x_test), loss, accuracy 

6. Instantiate the Cifar_client class defined above to run the client code.

  flower.client.start_numpy_client("[::]:8080", client=Cifar_client())

  The first argument in the above function call denotes the address of the server associated with the client. Here it is “[::]:8080” which means we can run both the server and the client on the same machine.

Flower Server

Now create a server file say serv.py as follows:

1. Import the Flower framework

import flwr as flower

2. Start the server

flower.server.start_server(config={"num_rounds": 3})

Run the federated learning system as follows after preparing the client and server files.

1. Run the server file first

python serv.py

2. Open another terminal and run the client file

python cli.py

If we have multiple clients as is the case in most of the practical applications, run the above command in different terminals – one for each client.

Source: https://github.com/adap/flower/blob/main/examples/quickstart_tensorflow/client.py

Advantages of Flower framework

• It was designed to work with large number of clients (recorded count of clients till date exceeds 10,000)
• It is compatible with a variety of ML frameworks like Keras and PyTorch.
• It supports a wide range of devices and servers including Android, Nvidia Jetson, iOS and Raspberry Pi.
• It is platform independent.
• It is an easy-to-use framework not requiring coding of unmanageable length.

EndNote

Federated Learning provides a collaborative way of implementing Machine Learning models with no need of centralizing the training data and model’s updates. It enables real-time predictions as users can interact with the model-handling server from mobile devices. The Flower framework allows building efficient federal learning systems across numerous decentralized devices or servers having local data and hence leveraging the Federal Learning approach.

Refer to the following links to get deeper insights of the Flower framework:

• Official website
• Documentation
• GitHub