Information processing device and information processing method
There is provided an information processing device which efficiently executes machine learning. The information processing device according to one embodiment includes: an obtaining unit which obtains a source code including a code which defines Forward processing of each layer constituting a neural network; a storage unit which stores an association relationship between each Forward processing and Backward processing associated with each Forward processing; and an executing unit which successively executes each code included in the source code, and which calculates an output value of the Forward processing defined by the code based on an input value at a time of execution of each code, and generates a reference structure for Backward processing in a layer associated with the code based on the association relationship stored in the storage unit.
1. An information processing device comprising:
at least one memory; and
at least one processor configured to train a neural network using a define-by-run configuration.
2. The information processing device according to claim 1 , wherein using the define-by-run configuration, the at least one processor is configured to:
execute forward processing of the neural network;
generate, by the execution of the forward processing, a calculation procedure for backward processing of the neural network;
execute, based on the generated calculation procedure, the backward processing of the neural network; and
update, based on a result of the backward processing, one or more parameters of the neural network.
3. The information processing device according to claim 2 , wherein the generated calculated procedure is represented by a graph.
4. The information processing device according to claim 3 , wherein the graph is not constructed before the execution of the forward processing.
5. The information processing device according to claim 3 , wherein the at least one processor is configured to dynamically construct the graph by the execution of the forward processing.
6. The information processing device according to claim 3 , wherein
the executing the forward processing of the neural network includes providing input data into the neural network and
changing the graph based on a property of the input data.
7. The information processing device according to claim 3 , wherein
the graph represents a series of processes of inputting data into the neural network, obtaining an output from the neural network and comparing the output with a correct answer.
8. The information processing device according to claim 2 , wherein the at least one processor is configured to execute the backward processing in reverse order of the forward processing based on the generated calculated procedure.
9. The information processing device according to claim 1 , wherein using the define-by-run configuration, the at least one processor is configured to dynamically construct a graph to train the neural network.
10. The information processing device according to claim 1 , wherein using the define-by-run configuration, the at least one processor is configured to simultaneously execute forward processing of the neural network and generate a calculation procedure for backward processing of the neural network.
11. An information processing system comprising:
a first device comprising at least one first memory and at least one first processor; and
a second device comprising at least one second memory and at least one second processor,
wherein the at least one first processor is configured to train a neural network using a define-by-run configuration,
wherein the at least one second processor is configured to generate output data by inputting input data into the neural network trained by the first device.
12. The information processing system according to claim 11 , wherein using the define-by-run configuration, the at least one first processor is configured to:
execute forward processing of the neural network;
generate, by the execution of the forward processing, a calculation procedure for backward processing of the neural network;
execute, based on the generated calculation procedure, the backward processing of the neural network; and
update, based on a result of the backward processing, one or more parameters of the neural network.
13. The information processing system according to claim 11 , wherein using the define-by-run configuration, the at least one first processor is configured to dynamically construct a graph to train the neural network.
14. The information processing system according to claim 11 , wherein using the define-by-run configuration, the at least one first processor is configured to simultaneously execute forward processing of the neural network and generate a calculation procedure for backward processing of the neural network.
15. A method comprising:
storing, in at least one memory, a neural network; and
training, by at least one processor, the neural network using a define-by-run configuration.
16. The method according to claim 15 , wherein the training the neural network using the define-by-run configuration comprises:
executing forward processing of the neural network;
generating, by the execution of the forward processing, a calculation procedure for backward processing of the neural network;
executing, based on the generated calculation procedure, the backward processing of the neural network; and
updating, based on a result of the backward processing, one or more parameters of the neural network.
17. The method according to claim 16 , wherein the generated calculated procedure is represented by a graph.
18. The method according to claim 17 , wherein the graph is not constructed before the execution of the forward processing.
19. The method according to claim 17 , wherein the training the neural network using the define-by-run configuration comprises:
dynamically constructing the graph by the execution of the forward processing.
20. The method according to claim 17 , wherein the executing the forward processing of the neural network includes
providing input data into the neural network, and
changing the graph based on a property of the input data.
21. The method according to claim 17 , wherein
the graph represents a series of processes of inputting data into the neural network, obtaining an output from the neural network, and comparing the output with a correct answer.
22. The method according to claim 16 , wherein the training the neural network using the define-by-run configuration comprises:
executing the backward processing in reverse order of the forward processing based on the generated calculated procedure.
23. The method according to claim 15 , wherein the training the neural network using the define-by-run configuration comprises:
dynamically constructing a graph to train the neural network.
24. The method according to claim 15 , wherein the training the neural network using the define-by-run configuration comprises:
simultaneously executing forward processing of the neural network and generating a calculation procedure for backward processing of the neural network.
25. A non-transitory computer readable medium storing program instructions for causing at least one processor to:
store, in at least one memory, a neural network; and
train the neural network using a define-by-run configuration.
26. The non-transitory computer readable medium according to claim 25 , wherein using the define-by-run configuration, the at least one processor is configured to:
execute forward processing of the neural network;
generate, by the execution of the forward processing, a calculation procedure for backward processing of the neural network;
execute, based on the generated calculation procedure, the backward processing of the neural network; and
update, based on a result of the backward processing, one or more parameters of the neural network.
27. The non-transitory computer readable medium according to claim 26 , wherein the generated calculated procedure is represented by a graph.
28. The non-transitory computer readable medium according to claim 27 , wherein the graph is not constructed before the execution of the forward processing.
29. The non-transitory computer readable medium according to claim 27 , wherein the at least one processor is configured to dynamically construct the graph by the execution of the forward processing.
30. The non-transitory computer readable medium according to claim 27 , wherein the executing the forward processing of the neural network includes
providing input data into the neural network, and
changing the graph based on a property of the input data.
31. The non-transitory computer readable medium according to claim 27 , wherein
the graph includes a series of processes of inputting data into the neural network, obtaining an output from the neural network, and comparing the output with a correct answer.
32. The non-transitory computer readable medium according to claim 26 , wherein the at least one processor is configured to execute the backward processing in reverse order of the forward processing based on the generated calculated procedure.
33. The non-transitory computer readable medium according to claim 25 , wherein using the define-by-run configuration, the at least one processor is configured to dynamically construct a graph to train the neural network.
34. The non-transitory computer readable medium according to claim 25 , wherein using the define-by-run configuration, the at least one processor is configured to simultaneously execute forward processing of the neural network and generate a calculation procedure for backward processing of the neural network.