Federated ensemble learning from decentralized data with incremental and decremental updates

1 . A computer implemented method comprising:

distributing a plurality of prediction models in a federated ensemble learning environment, where each of a plurality of clients initially includes at least one associated prediction model from the plurality of prediction models, among all of the plurality of clients to provide each of the plurality of clients with each of the plurality of prediction models;

evaluating each of the plurality of prediction models on at least a portion of a local dataset resident on each of the plurality of clients to output a quantification indicating how each of the plurality of prediction models fits at least the portion of the local dataset of each of the plurality of clients; and

generating an ensemble model by a central server by applying weights to each of the plurality of prediction models based on evaluation results computed by each client locally, the evaluation results including a value, a gradient, and a Hessian matrix of a user-defined objective,

wherein the ensemble model is generated in a decentralized manner without including an exchange of raw data among the plurality of clients.

2 . The computer implemented method of claim 1 , wherein a vector is independent of a size of the local dataset resident on each of the plurality of clients.

3 . The computer implemented method of claim 1 , further comprising performing one or more rounds of distributed gradient descent on the weights.

4 . The computer implemented method of claim 1 , further comprising limiting a number of models from the plurality of prediction models that are assigned a weight greater than zero to a predetermined threshold.

5 . The computer implemented method of claim 1 , further comprising excluding a number of clients, and additional associated prediction models resident on respective ones of the number of clients, from the plurality of clients.

6 . The computer implemented method of claim 1 , further comprising:

adding an additional client to the plurality of clients;

distributing each of the plurality of prediction models to the additional client; and

distributing one or more additional models associated with the additional client to the plurality of clients such that the one or more additional models is evaluated on the local dataset resident on each of the plurality of clients, and each of the plurality of prediction models, including the one or more additional models, are evaluated on the additional client.

7 . The computer implemented method of claim 1 , further comprising, removing or ignoring the associated prediction model of a removed client from each of the plurality of clients.

8 . The computer implemented method of claim 1 , further comprising, upon determining that a model has changed on one of the plurality of clients, re-valuating the changed model on each of the plurality of clients on at least the portion of the local dataset resident on each of the plurality of clients.

9 . The computer implemented method of claim 1 , further comprising, upon determining that at least the portion of the local dataset of a changed client of the plurality of clients is changed, re-evaluating each of the plurality of prediction models on at least the portion of the local dataset of the changed client.

10 . The computer implemented method of claim 1 , further comprising optimizing the weights applied to each of the plurality of prediction models to minimize error between a predicted label given by the ensemble model and a ground truth label.

11 . The computer implemented method of claim 1 , further comprising sending to the central server, each vector for each of the plurality of clients.

12 . A computerized federated ensemble learning system comprising:

a plurality of clients in communication with a server;

a plurality of prediction models resident at each of the plurality of clients; and

a local dataset resident at each of the plurality of clients, wherein:

each of the plurality of prediction models are distributed among each of the plurality of clients,

each of the plurality of clients is configured to evaluate each of the plurality of prediction models on at least a portion of the local dataset resident on each of the plurality of clients and output a quantification indicating how each of the plurality of prediction models fits at least the portion of the local dataset of each of the plurality of clients, and

the server is configured to receive a vector from each of the plurality of clients and generate an ensemble model by applying weights to each of the plurality of prediction models, wherein:

the ensemble model is generated in a decentralized manner without including an exchange of raw data among the plurality of clients, and

the ensemble model is configured to allow incremental and/or decremental updates to one or more of the plurality of clients without a need to re-evaluate all of the plurality of prediction models.

13 . The computerized federated ensemble learning system of claim 12 , wherein the vector is independent of a size of the local dataset resident at each of the plurality of clients.

14 . The computerized federated ensemble learning system of claim 12 , wherein the server is configured to assign the weights such that a predetermined threshold number of models from the plurality of prediction models are assigned a weight greater than zero.

15 . A non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions that, when executed, causes a computer device to carry out a method of improving computing efficiency of a computing device operating a federated learning system, the method comprising:

distributing a plurality of prediction models in a federated ensemble learning environment, where each of a plurality of clients initially includes at least one associated prediction model from the plurality of prediction models, among all of the plurality of clients to provide each of the plurality of clients with each of the plurality of prediction models;

evaluating each of the plurality of prediction models on at least a portion of a local dataset resident on each of the plurality of clients to output a quantification indicating how each of the plurality of prediction models fits at least the portion of the local dataset of each of the plurality of clients; and

generating an ensemble model by a central server by applying weights to each of the plurality of prediction models based on evaluation results computed by each client locally, the evaluation results including a value, a gradient, and a Hessian matrix of a user-defined objective,

wherein the ensemble model is generated in a decentralized manner without including an exchange of raw data among the plurality of clients.

16 . The non-transitory computer readable storage medium of claim 15 , wherein a vector is independent of a size of the local dataset resident on each of the plurality of clients.

17 . The non-transitory computer readable storage medium of claim 15 , wherein the execution of the computer readable program code further configures the computer device to perform operations comprising limiting a number of models from the plurality of prediction models that are assigned a weight greater than zero to a predetermined threshold.

18 . The non-transitory computer readable storage medium of claim 15 , wherein the execution of the computer readable program code further configures the computer device to perform operations comprising optimizing the weights applied to each of the plurality of prediction models to minimize error between a predicted label given by the ensemble model and a ground truth label.