Model training by discarding relatively less relevant parameters
Embodiments of the present invention include a system, computer-implemented method, and a computer program product. A non-limiting example of the method includes a processor utilizing a model having a plurality of parameters. The processor compares a current value of a model parameter to a prior value of the model parameter. Based at least in part on comparing the current value of the model parameter to the prior value of the model parameter, a determination is made that the model being utilized by the processor has changed. The current value of the model parameter is transmitted by the processor.
1. A computer-implemented method comprising:
training, by a processor of a base computer, an artificial neural network model to update one or more of a plurality of parameters of the artificial neural network model on the base computer, wherein the plurality of parameters including a first parameter and a second parameter;
comparing, by the processor, a first current value of the first parameter from a current training iteration to a first prior value of the first parameter from a prior training iteration and a second current value of the second parameter from the current training iteration to a second prior value of the second parameter from the prior training iteration;
determining, by the processor, to transmit the first current value of the first parameter to a remote device based at least in part on comparing the first current value of the first parameter and the first prior value of the first parameter, wherein a change between the first current value and the first prior value of the first parameter is more than a threshold value;
determining, by the processor, to not transmit the second current value of the second parameter to the remote device based at least in part on comparing the second current value of the second parameter and the second prior value of the second parameter, wherein a different change between the second current value and the second prior value of the second parameter is less than the threshold value; and
transmitting, by the processor, the first current value of the first parameter to the remote device to update the first parameter of a version of the artificial neural network model that is implemented on the remote device but is not trained by the remote device.
2. The computer-implemented method of claim 1 , wherein determining, by the processor, to transmit the first current value of the first parameter to the remote device further comprises determining, by the processor, a qualitative or quantitative measure over time of learning by the version of the artificial neural network model on the remote device.
3. The computer-implemented method of claim 1 , wherein the threshold value is a fixed value.
4. The computer-implemented method of claim 1 , wherein the threshold value is a changing value.
5. The computer-implemented method of claim 1 , wherein the artificial neural network model is a deep neural network, a convolutional neural network, or a deep belief network.
6. The computer-implemented method of claim 1 , wherein the processor is located in a cloud computing environment.
7. A system comprising:
a memory storing an artificial neural network model; and
a processor configured to:
train, at a base computer, an artificial neural network to update one or more of a plurality of parameters of the artificial neural network model on the base computer, wherein the plurality of the parameters includes a first parameter and a second parameter;
compare a first current value of the first parameter from a current training iteration to a first prior value of the first parameter from a prior training iteration and a second current value of the second parameter from the current training iteration to a second prior value of the second parameter from the prior training iteration;
determine to transmit the first current value of the first parameter to a remote device, based at least in part on a comparison of the first current value of the first parameter and the first prior value of the first parameter, wherein a change between the first current value and the first prior value of the first parameter is more than a threshold value;
determine to not transmit the second current value of the second parameter to the remote device based at least in part on comparing the second current value of the second parameter and the second prior value of the second parameter, wherein a different change between the second current value and the second prior value of the second parameter is less than the threshold value; and
transmit the first current value of the first parameter to the remote device to update the first parameter of a version of the artificial neural network model that is implemented on the remote device but is not trained by the remoted device.
8. The system of claim 7 , wherein the processor is further configured to determine to transmit the first current value of the first parameter to the remote device based on determining a qualitative or quantitative measure over time of learning by the version of the artificial neural network model on the remote device.
9. The system of claim 7 , wherein the threshold value is a fixed value.
10. The system of claim 7 , wherein the threshold value is a changing value.
11. The system of claim 7 , wherein the artificial neural network model is a deep neural network, a convolutional neural network, or a deep belief network.
12. The system of claim 7 , wherein the processor is located in a cloud computing environment.
13. A computer program product comprising:
a storage medium, readable by a processing circuit, storing program instructions for execution by the processing circuit for performing a method comprising:
training, at a base computer, an artificial neural network to update one or more of a plurality of parameters of the artificial neural network model on the base computer, wherein the plurality of parameters includes a first parameter and a second parameter;
comparing a first current value of the first parameter from a current training iteration to a first prior value of the first parameter from a prior training iteration and a second current value of the second parameter from the current training iteration to a second prior value of the second parameter from the prior training iteration;
determining to transmit the first current value of the first parameter to a remote device, based at least in part on comparing the first current value of the first parameter and the first prior value of the first parameter, wherein a change between the first current value and the first prior value of the first parameter is more than a threshold value;
determining to not transmit the second current value of the second parameter to the remote device based at least in part on comparing the second current value of the second parameter and the second prior value of the second parameter, wherein a different change between the second current value and the second prior value of the second parameter is less than the threshold value; and
transmitting the first current value of the first parameter to the remote device to update the first parameter of a version of the artificial neural network model that is implemented on the remote device but is not trained by the remote device.
14. The computer program product of claim 13 , wherein determining to transmit the first current value of the first parameter to the remote device further comprises determining a qualitative or quantitative measure over time of learning by the version of the artificial neural network model on the remote device.
15. The computer program product of claim 13 , wherein the threshold value is a fixed value or a changing value.
16. The computer program product of claim 13 , wherein the artificial neural network model is a deep neural network, a convolutional neural network, or a deep belief network.
17. The computer program product of claim 13 , wherein the processing circuit and the storage medium are located in a cloud computing environment.