Information processing apparatus and method
According to one embodiment, an apparatus includes a processor and a memory. The processor performs a learning process of a neural network including a batch normalization layer. The processor sets up the neural network. The processor updates, in the learning process, a weight parameter and a normalization parameter, used in the normalization of the batch normalization layer, alternately or at different timings.
1. An information processing apparatus comprising:
a processor configured to perform a learning process of a neural network including a batch normalization layer; and
a memory configured to be used in the learning process performed by the processor,
wherein the processor is configured to:
set up the neural network;
perform, in the learning process, an update of a weight parameter and an update of a normalization parameter used in normalization of the batch normalization layer at different timings based on a predetermined condition; and
perform a fixing of the weight parameter and the update of the normalization parameter in response to a learning accuracy of the neural network being lowered below a threshold after performing the update of the weight parameter.
2. The apparatus of claim 1 , wherein the processor is configured to switch between the update of the weight parameter and the update of the normalization parameter each time a predetermined learning process is repeated.
3. The apparatus of claim 1 , wherein the update of the normalization parameter includes adjusting a parameter for affine transformation of an output of the normalization.
4. The apparatus of claim 1 , wherein the processor is configured to perform the update of the weight parameter and the update of the normalization parameter at different timings, while a back propagation process of an error included in the learning process of the neural network is performed.
5. The apparatus of claim 1 , wherein the neural network includes:
a first stage layer including a first convolution layer and a first batch normalization layer; and
a second stage layer including a second convolution layer and a second batch normalization layer.
6. A method for a learning process of a neural network including a batch normalization layer, the method comprising:
setting up the neural network; and
performing, in the learning process, an update of a weight parameter and an update of a normalization parameter used in normalization of the batch normalization layer at different timings based on a predetermined condition, the performing including performing a fixing of the weight parameter and the update of the normalization parameter in response to a learning accuracy of the neural network being lowered below a threshold after performing the update of the weight parameter.
7. The method of claim 6 , wherein the performing includes switching between the update of the weight parameter and the update of the normalization parameter each time a predetermined learning process is repeated.
8. The method of claim 6 , wherein the update of the normalization parameter includes adjusting a parameter for affine transformation of an output of the normalization.
9. The method of claim 6 , wherein the performing includes performing the update of the weight parameter and the update of a normalization parameter at different timings, while a back propagation process of an error included in the learning process of the neural network is performed.
10. The method of claim 6 , wherein the neural network includes:
a first stage layer including a first convolution layer and a first batch normalization layer; and
a second stage layer including a second convolution layer and a second batch normalization layer.