Device, method and program for erasing select contributing layers of a neural network

1. A learning device, comprising:

a memory; and

a processor, coupled to the memory, configured to:

calculate, for each layer of a plurality of layers within a multilayer neural network, a degree of contribution based on a norm of a non-linear mapping corresponding to each layer indicating a degree of contribution to an estimation result of the multilayer neural network, wherein the multilayer neural network is a residual network and the plurality of layers include residual units that are stacked to construct the multilayer neural network;

select one or more to-be-erased layers from the plurality of layers within the multilayer neural network based on comparing the degree of contribution of each layer of the multilayer neural network and selecting a predetermined number of layers having lower degrees of contribution than all other layers not selected;

erase the one of more to-be-erased layers from the multilayer neural network by erasing the non-linear mapping from residual units of the multilayer neural network corresponding to the one or more to-be-erased layers;

retrain the multilayer neural network from which the one or more to-be-erased layers have been erased, wherein the learning device performs estimation with decreased memory consumption after the one or more to-be-erased layers have been erased; and

perform image recognition using the retrained multilayer neural network.

2. The learning device according to claim 1 , wherein the processor selects a user-defined predetermined number of layers as the one or more to-be-erased layers.

3. The learning device according to claim 1 , wherein the one or more to-be-erased layers are selected based on an absolute value of the degree of contribution of each layer of the plurality of layers.

4. The learning device according to claim 1 , wherein a last layer of the multilayer neural network includes a softmax function.

5. The learning device according to claim 1 , wherein a last layer of the multilayer neural network includes a cross entropy error function.

6. The learning device according to claim 4 , wherein erasure of the one or more to-be-erased layers result in passage of an input to the one or more to-be-erased layers to a next layer in the multilayer neural network.

7. The learning device according to claim 6 , wherein the norm includes a Frobenius norm.

8. The learning device according to claim 6 , wherein the norm includes a maximum norm.

9. The learning device according to claim 7 , wherein the processor uses stochastic gradient descent to retrain the multilayer neural network.

10. The learning device according to claim 7 , wherein the processor repeatedly performs the calculation of the degree of contribution, the selection of the one or more to-be-erased layers, the erasure of the one or more to-be-erased layers, and the retraining of the multilayer neural network until the multilayer neural network reaches a target number of layers of the plurality of layers.

11. A learning method to be executed by a learning device, comprising:

calculating, for each layer of a plurality of layers within a multilayer neural network, a degree of contribution based on a norm of a non-linear mapping corresponding to each layer indicating a degree of contribution to an estimation result of the multilayer neural network, wherein the multilayer neural network is a residual network and the plurality of layers include residual units that are stacked to construct the multilayer neural network;

selecting one or more to-be-erased layers from the plurality of layers of the multilayer neural network based on comparing the degree of contribution of each layer of the plurality of layers and selecting a predetermined number of layers having lower degrees of contribution than all other layers not selected;

erasing the one or more to-be-erased layers from the multilayer neural network by erasing the non-linear mapping from residual units of the multilayer neural network corresponding to the one or more to-be-erased layers;

retraining the multilayer neural network from which the one or more to-be-erased layers have been erased, wherein the learning device performs estimation with decreased memory consumption after the one or more to-be-erased layers have been erased; and

performing image recognition using the retrained multilayer neural network.

12. A non-transitory computer-readable medium encoded with a learning program that, when executed by a computer, cause the computer to perform a method comprising:

calculating, for each layer of a plurality of layers within a multilayer neural network, a degree of contribution based on a norm of a non-linear mapping corresponding to each layer indicating a degree of contribution to an estimation result of the multilayer neural network, wherein the multilayer neural network is a residual network, and the plurality of layers include residual units that are stacked to construct the multilayer neural network;

selecting one or more to-be-erased layers from the plurality of layers of the multilayer neural network based on comparing the degree of contribution of each layer of the plurality of layers and selecting a predetermined number of layers having lower degrees of contribution than all other layers not selected;

erasing the one or more to-be-erased layers from the multilayer neural network by erasing the non-linear mapping from residual units of the multilayer neural network corresponding to the one or more to-be-erased layers;

retraining the multilayer neural network from which the one or more to-be-erased layers have been erased, wherein the learning device performs estimation with decreased memory consumption after the one or more to-be-erased layers have been erased; and

performing image recognition using the retrained neural network.