Method, device, and computer program product for deep learning
A method, device and computer program product for deep learning are provided. According to one example, a parameter related to a deep learning model for a training dataset allocated to a server is obtained at a client; a transmission state of the parameter is determined, the transmission state indicating whether the parameter has been transmitted to the server; and information associated with the parameter to be sent to the server is determined based on the transmission state to update the deep learning model. Therefore, the performance of deep learning may be improved, and the network load of deep leaning may be reduced.
1. A method for deep learning, comprising:
obtaining, at a client, a parameter related to a deep learning model for a training dataset allocated to a server;
determining a transmission state of the parameter, the transmission state indicating whether the parameter has been transmitted to the server; and
determining, based on the transmission state, information associated with the parameter to be sent to the server to update the deep learning model;
wherein determining the transmission state comprises:
generating a first digest for identifying the parameter;
obtaining a predetermined second digest associated with the parameter; and
comparing the first digest with the predetermined second digest to determine whether the transmission state indicates that the parameter has been transmitted.
2. The method of claim 1 , wherein obtaining the parameter comprises:
obtaining a weight related to the deep learning model for the training dataset, the weight being determined based on a weight change from the server.
3. The method of claim 1 , wherein in response to the first digest matching the predetermined second digest, determining that the transmission state indicates the parameter has been transmitted; and
in response to the first digest mismatching the predetermined second digest, determining that the transmission state indicates the parameter has not been transmitted.
4. The method of claim 1 , wherein determining the information comprises:
in response to the transmission state indicating the parameter has been transmitted, sending, to the server, an identifier related to the parameter and the transmission state; and
in response to the transmission state indicating the parameter has not been transmitted, sending, to the server, the identifier, the transmission state and the parameter.
5. The method of claim 1 , further comprising:
deleting the predetermined second digest at the client; and
storing the first digest at the client.
6. A computer program product, tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when executed, cause a machine to implement the method according to claim 1 .
7. A device for deep learning, comprising:
at least one processing unit;
at least one memory, coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform acts comprising:
obtaining, at a client, a parameter related to a deep learning model for a training dataset allocated to a server;
determining a transmission state of the parameter, the transmission state indicating whether the parameter has been transmitted to the server; and
determining, based on the transmission state, information associated with the parameter to be sent to the server to update the deep learning model;
wherein determining the transmission state comprises:
generating a first digest for identifying the parameter;
obtaining a predetermined second digest associated with the parameter; and
comparing the first digest with the predetermined second digest to determine whether the transmission state indicates that the parameter has been transmitted.
8. The device of claim 7 , wherein obtaining the parameter comprises:
obtaining a weight related to the deep learning model for the training dataset, the weight being determined based on a weight change from the server.
9. The device of claim 7 , wherein in response to the first digest matching the predetermined second digest, determining that the transmission state indicates the parameter has been transmitted; and
in response to the first digest mismatching the predetermined second digest, determining that the transmission state indicates the parameter has not been transmitted.
10. The device of claim 7 , wherein determining the information comprises:
in response to the transmission state indicating the parameter has been transmitted, sending, to the server, an identifier related to the parameter and the transmission state; and
in response to the transmission state indicating the parameter has not been transmitted, sending, to the server, the identifier, the transmission state and the parameter.
11. The device of claim 7 , the acts further comprising:
deleting the predetermined second digest at the client; and
storing the first digest at the client.
12. A method for deep learning, comprising:
receiving, at a server, information associated with a first parameter and a transmission state of the first parameter from a client, the first parameter being related to a deep learning model for a training dataset allocated to the server, the transmission state indicating whether the first parameter has been transmitted to the server;
determining the first parameter based on the information; and
updating the deep learning model using the first parameter.
13. The method of claim 12 , wherein determining the first parameter comprises:
in response to determining that the first parameter has been transmitted based on the information, obtaining an identifier related to the first parameter from the information; and
obtaining, as the first parameter, a second parameter being previously stored and related to the first parameter based on the identifier.
14. The method of claim 12 , wherein determining the first parameter comprises:
in response to determining that the first parameter has not been transmitted based on the information, obtaining, from the information, the first parameter and an identifier related to the first parameter.
15. The method of claim 12 , further comprising:
deleting a second parameter at the server; and
storing the first parameter at the server.
16. A computer program product, tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when executed, cause a machine to implement the method according to claim 12 .
17. A device for deep learning, comprising at least one processing unit, at least one memory, coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform steps of the method according to claim 12 .
18. The device of claim 17 , wherein determining the first parameter comprises:
in response to determining that the first parameter has been transmitted based on the information, obtaining an identifier related to the first parameter from the information; and
obtaining, as the first parameter, a second parameter being previously stored and related to the first parameter based on the identifier.
19. The device of claim 17 , wherein determining the first parameter comprises:
in response to determining that the first parameter has not been transmitted based on the information, obtaining, from the information, the first parameter and an identifier related to the first parameter.
20. The device of claim 17 , the steps further comprising:
deleting a second parameter at the server; and
storing the first parameter at the server.