System and method for federated learning using weight anonymized factorization

1 . A client device in a federated machine-learning system, the client device comprising:

at least one computing device;

a communication interface; and

a processor coupled to the at least one computing device and to the communication interface, the processor configured to protect data on the client device through the federated machine-learning system by:

selecting a group of parameters for the client device from a global group of parameters,

training a client model using a dataset of the client device and the group of parameters selected by the client device,

updating a weight factor dictionary based in part on factors of a personalized matrix of the client device and comprising Bayesian nonparametric weight factorization values, and a factor strengths vector accounting for active weight factors, after training the client model, wherein the weight factor dictionary comprises factors of the weights used in the client model such that a weight is generated from a plurality of factors multiplied,

sending through the communication interface to a global server a client-updated weight factor dictionary and a client-updated factor strengths vector,

receiving through the communication interface from the global server a globally updated weight factor dictionary and a globally updated factor strengths vector, and

retraining the model using the dataset of the client device, the group of parameters selected by the client device, and the globally updated weight factor dictionary and the globally updated factor strengths vector so that the client model is personalized relative to those on another client device in the federated machine-learning system;

wherein the client device retains the selected group of parameters locally and transmits only derived update information in the form of the client-updated weight factor dictionary and the client-updated factor strengths vector; and

wherein neither the dataset of the client device nor any value of the selected group of parameters leave the client device, such that the global server receives no selected parameter values and is trained only from the locally computed update information to provide enhanced data security.

2 . The client device of claim 1 , wherein the client device is part of a group of N client devices in which N is an integer equal to 1 or more.

3 . The client device of claim 2 , wherein the processor selects the group of parameters from the global group of parameters by using three variational parameters that comprise seed values, and minimizes a difference between a supervised learning of the dataset and a regularization of the selected group of parameters and the global group of parameters.

4 . The client device of claim 3 , wherein the processor selects the group of parameters from the global group of parameters by receiving the global group of parameters that has been sent from the global server to a first subset of client devices of the N client devices, the client device being part of the first subset of client devices.

5 . The client device of claim 4 , wherein the client device receives the globally updated weight factor dictionary and a globally updated factor strengths vector by receiving the globally updated weight factor dictionary and a globally updated factor strengths vector that were sent by the global server to a second subset of the N client devices, the client device being part of the second subset of client devices.

6 . The client device of claim 4 ,

wherein the processor sends a request through the communication interface to the global server for a current version of the global group of parameters,

wherein the processor updates the model using the current version of the global group of parameters, and

wherein the processor evaluating the model updated using the current version of the global group of parameters to form an inference based on the dataset of the client device.

7 . The client device of claim 1 , wherein the dataset comprises information relating to at least one of biometric data, medical data, image data, voice data, location data, application-use data, thermal data, atmospheric data, audio data and survey data.

8 . A federated machine-learning system, comprising:

a global server that receives updates of weight factor dictionaries and factor strengths vectors from N client devices, in which N is an integer equal to 1 or more, and generates a globally updated weight factor dictionary and a globally updated factor strengths vector; and

the client devices, at least one client device comprising:

at least one computing device,

a communication interface, and

a processor coupled to the at least one computing device and to the communication interface, the processor configured to protect data on the client device through the federated machine-learning system by:

selecting a group of parameters from a global group of parameters,

training a client model using a dataset of the client device and the group of parameters selected by the client device,

updating a weight factor dictionary based in part on factors of a personalized matrix of the client device and comprising Bayesian nonparametric weight factorization values, and a factor strengths vector accounting for active weight factors, after training the client model, wherein the weight factor dictionary comprises factors of the weights used in the client model such that a weight is generated from a plurality of factors multiplied,

sending through the communication interface, a client-updated weight factor dictionary and a client-updated factor strengths vector,

receiving through the communication interface from the global server the globally updated weight factor dictionary and the globally updated factor strengths vector, and

retraining the model using the dataset of the client device, the group of parameters selected by the client device, and the globally updated weight factor dictionary and the globally updated factor strengths vector so that the model is personalized relative to those on another client device in the federated machine-learning system;

wherein the client device retains the selected group of parameters locally and transmits only derived update information in the form of the client-updated weight factor dictionary and the client-updated factor strengths vector; and

wherein neither the dataset of the client device nor any value of the selected group of parameters leave the client device, such that the global server receives no selected parameter values and is trained only from the locally computed update information to provide enhanced data security.

9 . The client device of claim 8 , wherein the processor selects the group of parameters from the global group of parameters by using three variational parameters that comprise seed values, and minimizes a difference between a supervised learning of the dataset and a regularization of the selected group of parameters and the global group of parameters.

10 . The client device of claim 9 , wherein the processor selects the group of parameters from the global group of parameters by receiving the global group of parameters that has been sent from the global server to a first subset of client devices of the N client devices, the client device being part of the first subset of client devices.

11 . The client device of claim 10 , wherein the client device receives the globally updated weight factor dictionary and a globally updated factor strengths vector by receiving the globally updated weight factor dictionary and a globally updated factor strengths vector that were sent by the global server to a second subset of the N client devices, the client device being part of the second subset of client devices.

12 . The client device of claim 10 ,

wherein the processor sends a request through the communication interface to the global server for a current version of the global group of parameters,

wherein the processor updates the model using the current version of the global group of parameters, and

wherein the processor evaluating the model updated using the current version of the global group of parameters to form an inference based on the dataset of the client device.

13 . The client device of claim 8 , wherein the dataset comprises information relating to at least one of biometric data, medical data, image data, voice data, location data, application-use data, thermal data, atmospheric data, audio data and survey data.

14 . A method for protecting data on a client device through a federated machine-learning system, the method comprising:

selecting, at the client device, a group of parameters from a global group of parameters for one or more jointly-learned components, the global group of parameters including a weight factor dictionary and a factor strengths vector;

training, at the client device, a client model using a dataset of the client device and the group of parameters selected by the client device;

updating the weight factor dictionary, based in part on factors of a personalized matrix of the client device and comprising Bayesian nonparametric weight factorization values, and the factor strengths vector accounting for active weight factors, after training the model, wherein the weight factor dictionary comprises factors of the weights used in the machine learning client model such that a weight is generated from a plurality of factors multiplied,

sending, from the client device to a global server, a client-updated weight factor dictionary and a client-updated factor strengths vector;

receiving, from the global server at the client device, a globally updated weight factor dictionary and a globally updated factor strengths vector; and

retraining, at the client device, the client model using the dataset of the client device, the group of parameters selected by the client device, and the globally updated weight factor dictionary and the globally updated factor strengths vector so that the client model is personalized relative to those on another client device in the federated machine-learning system;

wherein the client device retains the selected group of parameters locally and transmits only derived update information in the form of the client-updated weight factor dictionary and the client-updated factor strengths vector; and

wherein neither the dataset of the client device nor any value of the selected group of parameters leave the client device, such that the global server receives no selected parameter values and is trained only from the locally computed update information to provide enhanced data security.

15 . The method of claim 14 , wherein the client device is part of a group of N client devices in which N is an integer equal to 1 or more.

16 . The method of claim 15 , wherein selecting the group of parameters from the global group of parameters further comprises selecting the group of parameters using three variational parameters that comprise seed values; and

minimizing a difference between a supervised learning of the dataset and a regularization of the selected group of parameters and the global group of parameters.

17 . The method of claim 16 , wherein selecting the group of parameters from the global group of parameters further comprises receiving, at the client device, the global group of parameters that has been sent from the global server to a first subset of client devices of the N client devices, the client device being part of the first subset of client devices.

18 . The method of claim 17 , wherein receiving, from the global server at the client device, the globally updated weight factor dictionary and a globally updated factor strengths vector further comprises receiving, at the client device, the globally updated weight factor dictionary and a globally updated factor strengths vector that were sent by the global server to a second subset of the N client devices, the client device being part of the second subset of client devices.

19 . The method of claim 17 , further comprising:

requesting by the client device from the global server a current version of the global group of parameters;

receiving the current version of the global group of parameters;

updating the model using the current version of the global group of parameters; and

evaluating the model updated using the current version of the global group of parameters to form an inference based on the dataset of the client device.

20 . The method of claim 14 , wherein the dataset comprises information relating to at least one of biometric data, medical data, image data, voice data, location data, application-use data, thermal data, atmospheric data, audio data and survey data.