Updating machine learning models across devices

1 . A computer-implemented method comprising:

operating, by a first device, a first neural network model stored on the first device to determine output data, the first neural network model including a first plurality of layers;

determining at least a first updatable layer of the first plurality of layers to update based on training of the first neural network model;

determining one or more layers of the first neural network model, different from the first updatable layer, to be kept constant during training of the first neural network model;

training, by the first device, the first neural network model using the output data to determine first updated parameters for the first updatable layer, wherein parameters of the one or more layers are kept constant during the training;

sending, a second device, first model update data representing the first updated parameters;

receiving, from the second device in response to the first model update data, second model update data;

determining, using the second model update data, second updated parameters for a second updatable layer, the second updatable layer replacing the first updatable layer in the first neural network model to generate a second neural network model; and

operating, by the first device, the second neural network model using the second updatable layer in place of the first updatable layer.

2 . The computer-implemented method of claim 1 , further comprising:

operating, by the second device, a second neural network model, wherein the second neural network model includes a second plurality of layers and is configured to process a same type of data as the first neural network model;

receiving, by the second device, the first model update data;

receiving, by the second device, at least third model update data from at least a third device;

determining, using the first model update data and the third model update data, second updated parameters for one or more internal layers of the second plurality of layers;

determining, using the second updated parameters, the second model update data; and

sending the second model update data to the first device.

3 . A computer-implemented method comprising:

operating, by a first device, a first machine learning model stored on the first device to determine first output data;

determining first one or more layers of the first machine learning model to update based on training of the first machine learning model;

determining second one or more layers of the first machine learning model, different from the first one or more layers, to be kept constant during training of the first machine learning model;

training, by the first device, the first machine learning model using the first output data to determine first updated parameters for the first one or more layers, wherein parameters of the second one or more layers are kept constant during the training; and

sending, to a second device, first model update data representing the first updated parameters, the second device updating a second machine learning model based on the first model update data.

4 . The computer-implemented method of claim 3 , further comprising:

receiving, from the second device in response to the first model update data, second model update data;

determining, using the second model update data, second updated parameters for third one or more layers, the third one or more layers replacing the first one or more layers in the first machine learning model to generate a second machine learning model; and

operating, by the first device, the second machine learning model.

5 . The computer-implemented method of claim 3 , further comprising:

adding the first one or more layers between existing layers of a pre-trained machine learning model to generate the first machine learning model.

6 . The computer-implemented method of claim 3 , wherein:

determining the first one or more layers includes determining one or more existing layers of the first machine learning model to be the first one or more layers.

7 . The computer-implemented method of claim 3 , further comprising:

operating, by the second device, a second machine learning model, wherein the second machine learning model is configured to process a same type of data as the first machine learning model;

receiving, by the second device, the first model update data;

determining, using the first model update data, third updated parameters for one or more layers of the second machine learning model;

determining, using the third updated parameters, second model update data; and

sending the second model update data to the first device.

8 . The computer-implemented method of claim 3 , further comprising:

operating, by the second device, a second machine learning model, wherein the second machine learning model is configured to process a same type of data as the first machine learning model;

receiving, by the second device, the first model update data;

receiving, by the second device, at least third model update data from at least a third device;

determining, using the first model update data and the third model update data, second model update data; and

sending the second model update data to the first device.

9 . The computer-implemented method of claim 3 , further comprising:

receiving, from the second device in response to the first model update data, second model update data;

determining, using the second model update data, a second machine learning model representing a version of the first machine learning model updated based on the second model update data;

determining a dataset for validating the second model update data;

processing the dataset using the first machine learning model to determine second output data;

processing the dataset using the second machine learning model to determine third output data; and

determining, based at least in part on the first output data and the second output data, to process subsequent input data using the second machine learning model.

10 . The computer-implemented method of claim 3 , further comprising:

receiving, from the second device in response to the first model update data, second model update data;

determining, using the second model update data, a second machine learning model representing a version of the first machine learning model updated based on the second model update data;

determining a dataset for validating the second model update data;

processing the dataset using the first machine learning model to determine second output data;

processing the dataset using the second machine learning model to determine third output data;

determining, based at least in part on the second output data and the third output data, to perform a partial update of the first machine learning model; and

in response to determining to perform a partial update, determining, using the second model update data, a second machine learning model representing a partial update of the first machine learning model.

11 . The computer-implemented method of claim 3 , wherein:

generating the first model update data includes performing at least one mathematical function described by a protocol on the first updated parameters; and

performing the at least one mathematical function includes modifying one or more of a number of parameters or a value of one or more parameters of the first updated parameters.

12 . A system, comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:

operate, by a first device, a first machine learning model stored on the first device to determine first output data;

determine first one or more layers of the first machine learning model to update based on training of the first machine learning model;

determine second one or more layers of the first machine learning model, different from the first one or more layers, to be kept constant during training of the first machine learning model;

train, by the first device, the first machine learning model using the first output data to determine first updated parameters for the first one or more layers, wherein parameters of the second one or more layers are kept constant during the training; and

send, to a second device, first model update data representing the first updated parameters, the second device updating a second machine learning model based on the first model update data.

13 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive, from the second device in response to the first model update data, second model update data;

determine, using the second model update data, second updated parameters for third one or more layers, the third one or more layers replacing the first one or more layers in the first machine learning model to generate a second machine learning model; and

operate, by the first device, the second machine learning model.

14 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

add the first one or more layers between existing layers of a pre-trained machine learning model to generate the first machine learning model.

15 . The system of claim 12 , wherein determining the first one or more layers includes determining one or more existing layers of the first machine learning model to be the first one or more layers.

16 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

operate, by the second device, a second machine learning model, wherein the second machine learning model is configured to process a same type of data as the first machine learning model;

receive, by the second device, the first model update data;

determine, using the first model update data, third updated parameters for one or more layers of the second machine learning model;

determine, using the third updated parameters, second model update data; and

send the second model update data to the first device.

17 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

operate, by the second device, a second machine learning model, wherein the second machine learning model is configured to process a same type of data as the first machine learning model;

receive, by the second device, the first model update data;

receive, by the second device, at least third model update data from at least a third device;

determine, using the first model update data and the third model update data, second model update data; and

send the second model update data to the first device.

18 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive, from the second device in response to the first model update data, second model update data;

determine, using the second model update data, a second machine learning model representing a version of the first machine learning model updated based on the second model update data;

determine a dataset for validating the second model update data;

process the dataset using the first machine learning model to determine second output data;

process the dataset using the second machine learning model to determine third output data; and

determine, based at least in part on the first output data and the second output data, to process subsequent input data using the second machine learning model.

19 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive, from the second device in response to the first model update data, second model update data;

determine, using the second model update data, a second machine learning model representing a version of the first machine learning model updated based on the second model update data;

determine a dataset for validating the second model update data;

process the dataset using the first machine learning model to determine second output data;

process the dataset using the second machine learning model to determine third output data;

determine, based at least in part on the second output data and the third output data, to perform a partial update of the first machine learning model; and

in response to determining to perform a partial update, determine, using the second model update data, a second machine learning model representing a partial update of the first machine learning model.

20 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

generate the first model update data by performing at least one mathematical function described by a protocol on the first updated parameters, wherein the at least one mathematical function includes modifying one or more of a number of parameters or a value of one or more parameters of the first updated parameters.

21 . The computer-implemented method of claim 3 , further comprising:

determining loss data corresponding to determining the first output data, the first output data determined using first parameters of the first one or more layers;

performing a backpropagation algorithm using the loss data and the first parameters to determine gradient data corresponding to the first one or more layers; and

performing a gradient descent algorithm using the gradient data and the first parameters to determine the first updated parameters.

22 . The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine loss data corresponding to determining the first output data, the first output data determined using first parameters of the first one or more layers;

perform a backpropagation algorithm using the loss data and the first parameters to determine gradient data corresponding to the first one or more layers; and

perform a gradient descent algorithm using the gradient data and the first parameters to determine the first updated parameters.