Information processing apparatus and information processing method
[Problem] Effectively speeding up the learning using the DNN regardless of a learning technique. [Solution] An information processing apparatus is provided, which includes a learning unit that executes learning using a neural network, and the learning unit dynamically changes the value of the batch size during learning based on the gap value with the ideal state for learning output from the neural network. Furthermore, an information processing method is provided, which includes executing, by a processor, learning using a neural network, and the learning further includes dynamically changing the value of the batch size during learning based on the gap value with the ideal state for learning output from the neural network.
1. An information processing apparatus, comprising:
a memory; and
processing circuitry configured to execute learning using a neural network, wherein
the processing circuitry is further configured to dynamically change a value of a batch size during learning based on a gap value with an ideal state for learning output from the neural network, by controlling an increase or decrease in the batch size based on a value set by the processing circuitry, the controlling being performed by either
increasing or decreasing a number of calculation loops for learning to control an increase or decrease in the batch size, or
increasing or decreasing a number of GPUs used for learning to control an increase or decrease in the batch size.
2. The information processing apparatus according to claim 1 , wherein the gap value with the ideal state includes at least a loss.
3. The information processing apparatus according to claim 2 , wherein the processing circuitry increases the value of the batch size during learning when convergence of learning is expected based on the loss.
4. The information processing apparatus according to claim 3 , wherein the processing circuitry increases the value of the batch size during learning based on an n-th differential value of the loss.
5. The information processing apparatus according to claim 4 , wherein the processing circuitry increases the value of the batch size during learning based on whether at least any of the value of the loss and a gradient of the loss falls below a threshold.
6. The information processing apparatus according to claim 2 , wherein the processing circuitry decreases the value of the batch size during learning when divergence of learning is expected based on the loss.
7. The information processing apparatus according to claim 1 , wherein the processing circuitry dynamically changes the value of the batch size based on an epoch.
8. The information processing apparatus according to claim 7 , wherein the processing circuitry increases the value of the batch size due to progress of epochs.
9. The information processing apparatus according to claim 8 , wherein, when divergence of learning is expected based on the gap value with the ideal state, the processing circuitry reloads a network model in a previous epoch.
10. The information processing apparatus according to claim 9 , wherein, when the network model in the previous epoch is reloaded, the processing circuitry sets a value of the batch size smaller than a value set in the previous epoch.
11. The information processing apparatus according to claim 7 , wherein the processing circuitry increases the value of the batch size in each epoch.
12. The information processing apparatus according to claim 1 , wherein the processing circuitry reconstructs a model in a GPU to control an increase or decrease in the batch size.
13. An information processing apparatus, comprising:
a memory; and
processing circuitry configured to execute learning using a neural network, wherein
the processing circuitry is further configured to dynamically change a value of a batch size during learning based on a gap value with an ideal state for learning output from the neural network, by controlling an increase or decrease in the batch size based on a value set by the processing circuitry, and
when there is an additionally available GPU, the processing circuitry allocates the GPU to learning to control an increase in the batch size.
14. An information processing apparatus, comprising:
a memory; and
processing circuitry configured to execute learning using a neural network, wherein
the processing circuitry is further configured to dynamically change a value of a batch size during learning based on a gap value with an ideal state for learning output from the neural network, by controlling an increase or decrease in the batch size based on a value set by the processing circuitry, and
when there is no additionally available GPU and there is a free space in a memory of a currently used GPU, the processing circuitry reconstructs a model in the currently used GPU to control an increase in the batch size.
15. The information processing apparatus according to claim 1 , wherein, when there is no free space in a memory of a currently used GPU, the processing circuitry increases a number of calculation loops for learning to control the increase in the batch size.
16. The information processing apparatus according to claim 1 , wherein the gap value with the ideal state includes at least any of a training error and a validation error.