Learning processing device and learning processing method
A learning processing device and method achieves learning of a lightweight model that is completed in a short amount of time. The learning processing device obtains a new, second learning model from an existing first learning model. An input unit acquires a first learning model generated in advance by learning a first learning data set, and an unpruned neural network (hereinafter, NN). An important parameter identification unit uses the first learning model and the NN to initialize a NN to be learned, and uses a second learning data set and the initialized NN to identify a degree of importance of parameters in a recognition process of the initialized NN. A new model generation unit carries out a pruning process for deleting parameters which are not important from the initialized NN, thereby generating a second NN; and a learning unit uses the second learning data set to learn the second NN.
1 . A learning processing device for obtaining a new second learning model from an existing first learning model, the learning processing device comprising:
a central processing unit (CPU) configured to execute:
an input unit configured to acquire a first learning model generated in advance by learning a first learning data set, and an unpruned neural network;
an important parameter identification unit configured to initialize the neural network that is an object to be learned using the first learning model and the neural network, and to identify degrees of importance of parameters in recognition processing of the initialized neural network by using a second learning data set and the initialized neural network;
a new model generating unit that configured to generate a second neural network by performing pruning processing for deleting unimportant parameters from the initialized neural network using the degrees of importance of the parameters;
a learning unit configured to learn the second neural network using the second learning data set; and
an output unit configured to output the second neural network after learning as a second learning model,
wherein the degrees of importance include a sum of degrees of importance and a post-pruning sum of degrees of importance.
2 . The learning processing device according to claim 1 , further comprising:
a display unit, coupled to the CPU,
wherein the CPU is configured to:
obtain the degree of importance of the parameter obtained by the important parameter identification unit for each of the plurality of the first learning models,
store the degree of importance together with the type of the first learning model, and
to display, on the display unit, the degree of importance as degree-of-importance information.
3 . The learning processing device according to claim 1 ,
wherein the CPU is configured to execute:
a partial reinitialization unit configured to partially reinitialize the parameter to be pruned when the first learning model has not been able to learn a feature important for recognition processing of an object to be learned with respect to a degree of importance given by the important parameter identification unit, and
wherein processing of the new model generating unit is performed after the partial reinitialization.
4 . The learning processing device according to claim 1 ,
wherein the neural network is a convolutional neural network, and
wherein the CPU is configured to:
determine whether to use a feature channel of an output of a convolution layer for recognition in a some or all of the convolution layers included in the convolutional neural network is learned by relaxed Bernoulli distribution.
5 . The learning processing device according to claim 4 ,
wherein an amount of the feature channel used for recognition during learning of the relaxed Bernoulli distribution is gradually increased during the learning.
6 . The learning processing device according to claim 5 ,
wherein an evaluation of the degree of importance is obtained by an amount obtained from first-order or second-order differential information related to a parameter of the neural network of a loss function and the parameter of the neural network, or a parameter of the relaxed Bernoulli distribution.
7 . The learning processing device according to claim 6 ,
wherein in initializing a portion of the network structure that is not initialized by the parameter of the first learning model, the portion is initialized so that the parameter is easily pruned.
8 . A learning processing method for obtaining a new second learning model from an existing first learning model, the learning processing method comprising:
acquiring a first learning model generated in advance by learning a first learning data set, and an unpruned neural network;
initializing the neural network that is an object to be learned using the first learning model and the neural network;
identifying degrees of importance of parameters in recognition processing of the initialized neural network by using a second learning data set and the initialized neural network;
generating a second neural network by performing pruning processing for deleting unimportant parameters from the initialized neural network using the degrees of importance of the parameters;
learning the second neural network using the second learning data set;
and setting the second neural network after learning as a second learning model,
wherein the degrees of importance include a sum of degrees of importance and a post-pruning sum of degrees of importance.