Selective reporting of machine learning parameters for federated learning

1 . A system for facilitating distributed learning for a model by selectively indicating which layers of a neural network to send for updating the model, comprising:

one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

storing, in a network database, a federated machine learning model comprising a set of neural network weights, the set of neural network weights comprising a first layer of neural network weights and a second layer of neural network weights;

sending, via the Internet, a plurality of data selection parameters respectively to a plurality of client devices, the plurality of data selection parameters comprising (i) a first data selection parameter that corresponds to a first subset of layers and is sent to a first client device and (ii) a second data selection parameter that corresponds to a second subset of layers and is sent to a second client device and not to the first device;

receiving, via the Internet, (i) the first subset of layers of a first client-side version of the set of neural network weights, comprising the first layer of neural network weights and not comprising the second layer of neural network weights, from the first client device based on the first subset of layers corresponding to the first data selection parameter and (ii) the second subset of layers of a second client-side version of the set of neural network weights, comprising the second layer of neural network weights and not comprising the first layer of neural network weights, from the second client device based on the second subset of layers corresponding to the second data selection parameter; and

updating, in the network database, the federated machine learning model based on the first subset of layers of the first client-side version of the set of neural network weights and the second subset of layers of the second client-side version of the set of neural network weights.

2 . The system of claim 1 , wherein the first client device performs a training operation based on a set of user inputs, indicating user data entry and user selection perform at the first client device, to update the first client-side version of the set of neural network weights.

3 . The system of claim 1 , wherein sending the plurality of data selection parameters to the plurality of client devices comprises sending the plurality of data selection parameters to the plurality of client devices based on an update threshold being satisfied by a ratio derived from (i) a first count of versions of the first layer of neural network weights that are received from a first set of client devices and (ii) a second count of versions of the second layer of neural network weights that are received from a second set of client devices.

4 . The system of claim 1 , the operations further comprising:

sending a noise-related seed to the first client device in connection with sending the first data selection parameter corresponding to the first subset of layers,

wherein the first client device uses the noise-related seed to generate a set of noise values and updates the first subset of layers of the first client-side version based on the set of noise values.

5 . The system of claim 1 , the operations further comprising:

sending a restructuring order to the first client device in connection with sending the first data selection parameter corresponding to the first subset of layers, wherein, based on the restructuring order sent to the first client device, the first client device sorts parameters of the first subset of layers of the first client-side version into a resorted sequence of values; and

after receiving the first subset of layers of the first client-side version from the first client device, sorting the resorted sequence of values of the first subset of layers of the first client-side version,

wherein updating the federated machine learning model comprises updating the federated machine learning model based on (i) the sorting of the resorted sequence of values of the first subset of layers of the first client-side version and (ii) the second subset of layers of the second client-side version.

6 . A method comprising:

storing, in one or more databases, a machine learning model comprising a set of neural network weights, the set of neural network weights comprising a first subset of weights and a second subset of weights;

sending, via a network, a plurality of data selection parameters respectively to a plurality of client devices, the plurality of data selection parameters comprising (i) a first data selection parameter that identifies the first subset of weights and is sent to a first client device and (ii) a second data selection parameter that identifies a second subset of weights and is sent to a second client device and not to the first client device;

receiving, via a network, (i) the first subset of weights of a first client-side version of the set of neural network weights from the first client device based on the first subset of weights being identified by the first data selection parameter and (ii) the second subset of weights of a second client-side version of the set of neural network weights from the second client device based on the second subset of weights being identified by the second data selection parameter; and

updating, in the one or more databases, the machine learning model based on the first subset of weights of the first client-side version of the set of neural network weights and the second subset of weights of the second client-side version of the set of neural network weights.

7 . The method of claim 6 , wherein sending the plurality of data selection parameters to the plurality of client devices comprises sending, via a network, the first data selection parameter comprising one or more first identifiers corresponding to the first subset of weights to the first client device and (ii) the second data selection parameter comprising one or more second identifiers corresponding to the second subset of weights to the second client device.

8 . The method of claim 6 , further comprising:

sending a noise-related seed to the first client device,

wherein the first client device uses the noise-related seed to generate a set of noise values and updates the first subset of weights of the first client-side version based on the set of noise values.

9 . The method of claim 6 , further comprising:

sending a restructuring order to the first client device, wherein, based on the restructuring order sent to the first client device, the first client device sorts weights of the first subset of weights of the first client-side version into a resorted sequence of values; and

after receiving the first subset of weights of the first client-side version from the first client device, sorting the resorted sequence of values of the first subset of weights of the first client-side version,

wherein updating the machine learning model comprises updating the machine learning model based on (i) the sorting of the resorted sequence of values of the first subset of weights of the first client-side version and (ii) the second subset of weights of the second client-side version.

10 . The method of claim 6 , further comprising:

receiving, via a network, the first subset of weights of other client-side versions of the set of neural network weights from other client devices,

wherein updating the machine learning model comprises updating, in the one or more databases, the machine learning model based on (i) a combined parameter value derived from a measure of central tendency computed from the first subset of weights of the first client-side version and the first subset of weights of the other client-side versions and (ii) the second subset of weights of the second client-side version of the set of neural network weights.

11 . The method of claim 6 , further comprising:

receiving, via a network, the first subset of weights of other client-side versions of the set of neural network weights from other client devices,

wherein updating the machine learning model comprises updating, in the one or more databases, the machine learning model based on (i) a combined parameter value derived from one or more mean averages computed from the first subset of weights of the first client-side version and the first subset of weights of the other client-side versions and (ii) the second subset of weights of the second client-side version of the set of neural network weights.

12 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause operations comprising:

storing, in one or more databases, a machine learning model comprising a set of neural network weights, the set of neural network weights comprising a first subset of weights and a second subset of weights;

sending, via a network, to a first client device, a first data selection parameter that identifies the first subset of weights;

in connection with client-side training of a first client-side version of the set of neural network weights that updates the first subset of weights of the first client-side version and a second subset of weights of the first client-side version, receiving, via a network, from the first client device, based on the first subset of weights being identified by the first data selection parameter, a first subset of parameters corresponding to the first subset of weights of the first client-side version of the set of neural network weights, without receiving the updated second subset of weights that are updated from the client-side training from the first client device; and

updating, in the one or more databases, the machine learning model based on the first subset of parameters corresponding to the first subset of weights of the first client-side version of the set of neural network weights.

13 . The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:

sending a noise-related seed to the first client device,

wherein the first client device uses the noise-related seed to generate a set of noise values and updates the first subset of weights of the first client-side version based on the set of noise values.

14 . The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:

sending a restructuring order to the first client device, wherein, based on the restructuring order sent to the first client device, the first client device sorts parameters of the first subset of parameters into a resorted sequence of values; and

after receiving the first subset of weights of the first client-side version from the first client device, sorting the resorted sequence of values,

wherein updating the machine learning model comprises updating the machine learning model based on the sorting of the resorted sequence of values.

15 . The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:

receiving, via a network, other parameter subsets corresponding to the first subset of weights of other client-side versions of the set of neural network weights from other client devices,

wherein updating the machine learning model comprises updating, in the one or more databases, the machine learning model based on a combined parameter value derived from a measure of central tendency computed from the first subset of parameters and the other parameter subsets respectively corresponding to the first subset of weights of the first client-side version and the first subset of weights of the other client-side versions.

16 . The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:

receiving, via a network, the first subset of weights of other client-side versions of the set of neural network weights from other client devices,

wherein updating the machine learning model comprises updating, in the one or more databases, the machine learning model based on a combined parameter value derived from one or more mean averages computed from the first subset of weights of the first client-side version and the first subset of weights of the other client-side versions.

17 . The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:

receiving, via a network, from a second client device, a set of parameters corresponding to a second client-side version of the set of neural network weights; and

based on an accuracy threshold not being satisfied by a device-specific accuracy score derived from testing a version of the machine learning model, that incorporates the set of parameters, with an anonymized version of user input data from the first client device, associating parameters provided by the second client device with a flag indicating anomalous data.

18 . The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:

based on a count threshold not being satisfied by a first count of versions of the first subset of parameters that are received from a first set of client devices, generating a second machine learning model that does not include at least one layer level corresponding to at least one other layers of the machine learning model other than a first layer comprising the first subset of weights.

19 . The one or more non-transitory computer-readable media of claim 12 , wherein:

the first subset of parameters comprises difference values derived from respective differences between the first subset of weights of the first client-side version before a training update to the first client-side version and the first subset of weights of the first client-side version after the training update to the first client-side version, and

updating the machine learning model comprises updating, in the one or more databases, the machine learning model based on the difference values received from the first client device.