Framework for rapidly prototyping federated learning algorithms
A framework for rapidly prototyping federated learning algorithms. Specifically, the disclosed framework proposes a method and system for evaluating different hypotheses for configuring learning model, which may be optimized through federated learning. Through the disclosed framework, these hypotheses may be tested for scalability, hardware and network resource performance, as well as for new learning state compression and/or aggregation technique effectiveness. Further, these hypotheses may be tested through federated learning simulations, which avoid costs associated with deploying these hypotheses to be tested across production systems.
1. A system, comprising:
a model simulation service, comprising:
a plurality of computer processors,
wherein at least one computer processor of the plurality of computer processors is programmed to:
for each federated learning simulation of a plurality of federated learning simulations:
obtain a simulation configuration for the federated learning simulate on;
instantiate simulated nodes based on the simulation configuration;
emulate learning interactions between the simulated nodes;
probe the learning interactions to obtain a simulation analysis;
select, following a completion of the plurality of federated learning simulations, an optimal simulation configuration from a plurality of simulation configurations and based on a plurality of simulation analyses for the plurality of federated learning simulations,
a central node operatively connected to the model simulation service and comprising a computer processor,
a plurality of worker nodes operatively connected to the model simulation service and the central node, and each comprising a computer processor,
wherein the at least one computer processor of the model simulation service is further programmed to:
deploy a first portion of the optimal simulation configuration to the central node,
deploy a second portion of the optimal simulation configuration to each worker node of the plurality of worker nodes,
wherein the first portion of the optimal simulation configuration comprises an architecture for a global learning model through which the central node uses to participate in federated learning,
wherein the architecture for the global learning model comprises a plurality of learning model parameters and a plurality of learning model hyper-parameters, and
wherein the plurality of learning model parameters comprises a weight representing a connection strength between the simulated nodes.
2. The system of claim 1 , wherein the simulation configuration comprises a scenario configuration portion specifying a first collection of parameters that define an architecture of the federated learning simulation.
3. The system of claim 2 , wherein the simulation configuration further comprises an execution configuration portion specifying a second collection of parameters that govern an operation of the federated learning simulation.
4. The system of claim 1 , wherein the simulated nodes comprise a simulated central node and a plurality of simulated worker nodes programmed to participate in federated learning.
5. The system of claim 1 , wherein a learning state decompression technique for restoring compressed learning state submitted during federated learning into learning state is used by the central node to update the global learning model, and a learning rate defining a speed at which the central node updates the global learning model.
6. The system of claim 1 , wherein the second portion of the optimal simulation configuration comprises an architecture for a local learning model through which each worker node uses to participate in federated learning, a learning state compression technique for reducing a size of the learning state submitted to the central node during federated learning, a learning rate defining a speed at which each worker node updates the local learning model thereon, a first proportion size of local data to be used by each worker node to train the local learning model, and a second proportion size of local data to be used by each worker node to validate the local learning model.
7. A method for deploying an optimal federated learning system, comprising:
for each federated learning simulation of a plurality of federated learning simulations:
obtaining a simulation configuration for the federated learning simulation;
instantiating simulated nodes based on the simulation configuration;
emulating learning interactions between the simulated nodes;
probing the learning interactions to obtain a simulation analysis;
selecting, following a completion of the plurality of federated learning simulations, an optimal simulation configuration from a plurality of simulation configurations and based on a plurality of simulation analyses for the plurality of federated learning simulations; and
deploying the optimal federated learning system based on the optimal simulation configuration,
wherein the optimal federated learning system comprises a central node operatively connected to a plurality of worker nodes, wherein the central node and the plurality of worker nodes participate in federated learning, and
wherein deploying comprises:
deploying a first portion of the optimal simulation configuration to the central node,
deploying a second portion of the optimal simulation configuration to each worker node of the plurality of worker nodes,
wherein the first portion of the optimal simulation configuration comprises an architecture for a global learning model through which the central node uses to participate in federated learning,
wherein the architecture for the global learning model comprises a plurality of learning model parameters and a plurality of learning model hyper-parameters, and
wherein the plurality of learning model parameters comprises a weight representing a connection strength between the simulated nodes.
8. The method of claim 7 , wherein the first portion of the optimal simulation configuration comprises an architecture for a global learning model through which the central node uses to participate in federated learning, a learning state decompression technique for restoring compressed learning state submitted during federated learning into learning state used by the central node to update the global learning model, and a learning rate defining a speed at which the central node updates the global learning model.
9. The method of claim 7 , wherein the second portion of the optimal simulation configuration comprises an architecture for a local learning model through which each worker node uses to participate in federated learning, a learning state compression technique for reducing a size of the learning state submitted to the central node during federated learning, a learning rate defining a speed at which each worker node updates the local learning model thereon, a first proportion size of local data to be used by each worker node to train the local learning model, and a second proportion size of local data to be used by each worker node to validate the local learning model.
10. The method of claim 7 , wherein the simulation configuration comprises a scenario configuration portion specifying a first collection of parameters that define an architecture of the federated learning simulation.
11. The method of claim 10 , wherein the simulation configuration further comprises an execution configuration portion specifying a second collection of parameters that govern an operation of the federated learning simulation.
12. A non-transitory computer readable medium (CRM) comprising computer readable program code to deploy an optimal federated learning system, which when executed by at least one computer processor, enables the at least one computer processor to:
for each federated learning simulation of a plurality of federated learning simulations:
obtain a simulation configuration for the federated learning simulation;
instantiate simulated nodes based on the simulation configuration;
emulate learning interactions between the simulated nodes;
probe the learning interactions to obtain a simulation analysis;
select, following a completion of the plurality of federated learning simulations, an optimal simulation configuration from a plurality of simulation configurations and based on a plurality of simulation analyses for the plurality of federated learning simulations; and
deploy the optimal federated learning system based on the optimal simulation configuration,
wherein the optimal federated learning system comprises a central node operatively connected to a plurality of worker nodes, wherein the central node and the plurality of worker nodes participate in federated learning, and
wherein deploying comprises:
deploying a first portion of the optimal simulation configuration to the central node,
deploying a second portion of the optimal simulation configuration to each worker node of the plurality of worker nodes,
wherein the first portion of the optimal simulation configuration comprises an architecture for a global learning model through which the central node uses to participate in federated learning,
wherein the architecture for the global learning model comprises a plurality of learning model parameters and a plurality of learning model hyper-parameters, and
wherein the plurality of learning model parameters comprises a weight representing a connection strength between the simulated nodes.
13. The non-transitory CRM of claim 12 , wherein the first portion of the optimal simulation configuration comprises an architecture for a global learning model through which the central node uses to participate in federated learning, a learning state decompression technique for restoring compressed learning state submitted during federated learning into learning state used by the central node to update the global learning model, and a learning rate defining a speed at which the central node updates the global learning model.
14. The non-transitory CRM of claim 12 , wherein the second portion of the optimal simulation configuration comprises an architecture for a local learning model through which each worker node uses to participate in federated learning, a learning state compression technique for reducing a size of the learning state submitted to the central node during federated learning, a learning rate defining a speed at which each worker node updates the local learning model thereon, a first proportion size of local data to be used by each worker node to train the local learning model, and a second proportion size of local data to be used by each worker node to validate the local learning model.