System and method of federated learning with diversified feedback

1 . A computer-implemented method for federated learning in a network including a server and multiple client devices, comprising:

receiving, by the server, a set of parameters of a local machine-learning model from each client device in a subset of the multiple client devices;

combining, by the server, the set of parameters from each of the client devices in the subset to generate an integrated set of parameters;

calculating, by the server, a parameter difference between the integrated set of parameters and the set of parameters for each client device in the subset of the multiple client devices;

determining, by the server, whether the set of parameters from each of the client devices in the subset of multiple client devices are valid according to a Central Limit Theorem by calculating a first threshold value using a normal distribution based on the Central Limit Theorem and comparing the parameter difference to the first threshold value;

sending feedback to each client device in the subset of multiple client devices, the feedback being based on a comparison of the parameter difference of each client device in the subset of the multiple client devices to the first threshold value; and

when the parameter difference is greater than the first threshold value:

recording, by the server, a number of times the parameter difference for each of the client devices in the subset is greater than the first threshold value;

sending, by the server, the parameter difference to each of the client devices in the subset;

removing, by the server, the set of parameters for each of the client devices in the subset when the number of times is greater than a second threshold value;

recalculating, by the server, the integrated set of parameters to generate a recalculated integrated set of parameters; and

recalculating, by the server, a recalculated parameter difference between the recalculated integrated set of parameters and the set of parameters for each client device in the subset of the multiple client devices.

2 . The computer-implemented method of claim 1 , wherein removing the set of parameters for each of the client devices comprises removing the set of parameters collected from the client device from further calculation during a current iteration.

3 . The computer-implemented method of claim 1 , wherein the feedback includes the integrated set of parameters, a parameter difference normal distribution, and the parameter difference when the parameter difference is less than or equal to the first threshold value.

4 . The computer-implemented method of claim 3 , wherein the feedback is proportional to a standard deviation of the normal distribution.

5 . The computer-implemented method of claim 1 , further comprising:

setting, by the server, a particular client device as an outlier client device if the number of times is greater than the second threshold value; and

removing, by the server, the outlier client device from the subset.

6 . The computer-implemented method of claim 5 , wherein the setting includes determining the outlier client device based on the normal distribution and the first threshold value.

7 . The computer-implemented method of claim 1 , further comprising:

storing a global machine-learning model at the server; and

iteratively updating, by the server, the global machine-learning model based on the recalculated integrated set of parameters.

8 . A server for federated learning in a network having multiple client devices, comprising:

a non-transitory memory storing instructions; and

at least one processor in communication with the memory, the at least one processor configured, upon execution of the instructions, to perform the following steps:

receive a set of parameters of a local machine-learning model from each client device in a subset of the multiple client devices;

combine the set of parameters from each of the client devices in the subset to generate an integrated set of parameters;

calculate a parameter difference between the integrated set of parameters and the set of parameters for each client device in the subset of the multiple client devices; and

determine whether the set of parameters from each of the client devices in the subset of multiple client devices are valid according to a Central Limit Theorem by calculating a first threshold value using a normal distribution based on the Central Limit Theorem and comparing the parameter difference to the first threshold value;

send feedback to each client device in the subset of multiple client devices, the feedback being based on a comparison of the parameter difference of each client device in the subset of the multiple client devices to the first threshold value; and

when the parameter difference is greater than the first threshold value:

record a number of times the parameter difference for each of the client devices in the subset is greater than the first threshold value;

send the parameter difference to each of the client devices in the subset;

remove the set of parameters for each of the client devices in the subset when the number of times is greater than a second threshold value;

recalculate the integrated set of parameters to generate a recalculated integrated set of parameters; and

recalculate a recalculated parameter difference between the recalculated integrated set of parameters and the set of parameters for each client device in the subset of the multiple client devices.

9 . The server of claim 8 , wherein removing the set of parameters for each of the client devices comprises removing the set of parameters collected from the client device from further calculation during a current iteration.

10 . The server of claim 8 , wherein the feedback includes the integrated set of parameters, a parameter difference normal distribution, and the parameter difference when the parameter difference is less than or equal to the first threshold value.

11 . The server of claim 10 , wherein the feedback is proportional to a standard deviation of the normal distribution.

12 . The server of claim 8 , wherein for each client device in the subset for which the number is greater than the second threshold value, the one or more processors further execute the instructions to:

set a particular client device as an outlier client device if the number of times is greater than the second threshold value; and

remove the outlier client device from the subset.

13 . The server of claim 12 , wherein the one or more processors further execute the instructions to determine the outlier client device based on the normal distribution and the first threshold value.

14 . The server of claim 8 , the one or more processors further execute the instructions to:

store a global machine-learning model; and

iteratively update the global machine-learning model based on the recalculated integrated set of parameters.

15 . A non-transitory computer-readable media storing computer instructions for federated learning in a network including a server and multiple client devices, that configure at least one processor, upon execution of the instructions, to perform the following steps:

receiving a set of parameters of a local machine-learning model from each client device in a subset of the multiple client devices;

combining the set of parameters from each of the client devices in the subset to generate an integrated set of parameters;

calculating a parameter difference between the integrated set of parameters and the set of parameters for each client device in the subset of the multiple client devices;

determining whether the set of parameters from each of the client devices in the subset of multiple client devices are valid according to a Central Limit Theorem by calculating a first threshold value using a normal distribution based on the Central Limit Theorem and comparing the parameter difference to the first threshold value;

sending feedback to each client device in the subset of multiple client devices, the feedback being based on a comparison of the parameter difference of each client device in the subset of the multiple client devices to the first threshold value; and

when the parameter difference is greater than the first threshold value:

record a number of times the parameter difference for each of the client devices in the subset is greater than the first threshold value;

send the parameter difference to each of the client devices in the subset;

remove the set of parameters for each of the client devices in the subset when the number of times is greater than a second threshold value;

recalculate the integrated set of parameters to generate a recalculated integrated set of parameters; and

recalculate a recalculated parameter difference between the recalculated integrated set of parameters and the set of parameters for each client device in the subset of the multiple client devices.

16 . The non-transitory computer-readable medium of claim 15 , wherein removing the set of parameters for each of the client devices comprises removing the set of parameters collected from the client device from further calculation during a current iteration.

17 . The non-transitory computer-readable medium of claim 15 , wherein the feedback includes the integrated set of parameters, a parameter difference normal distribution, and the parameter difference when the parameter difference is less than or equal to the first threshold value.

18 . The non-transitory computer-readable medium of claim 17 , wherein the feedback is proportional to a standard deviation of the normal distribution.

19 . The non-transitory computer-readable medium of claim 15 , wherein for each client device in the subset for which the number is greater than the second threshold value, further causing the server to perform the steps of:

setting a particular client device as an outlier client device if the number of times is greater than the second threshold value; and

removing the outlier client device from the subset.

20 . The non-transitory computer-readable medium of claim 19 , wherein the setting includes determining the outlier client device based on the normal distribution and the first threshold value.

21 . The non-transitory computer-readable medium of claim 15 , further causing the server to perform the steps of:

storing a global machine-learning model; and

iteratively updating the global machine-learning model based on the set of parameters received from each of the client devices in the subset of the multiple client devices.