Learning apparatus and method

1 . A learning apparatus comprising a processor configured to:

train a neural network model having a plurality of pathways arranged between an input layer and an output layer, the neural network model including hidden layers with parameters comprising weight coefficients or channels and generate a trained model;

perform pruning on the trained model by calculating, for each pathway, an L2 norm of each channel in the hidden layer included in the pathway, and the number of channels having an L2 norm larger than a predetermined threshold;

generate a candidate model for reconstruction by deleting a pathway in which the number of channels having the L2 norm larger than the predetermined threshold is equal to or less than a threshold, and constructing the candidate model with the remaining pathways;

determine whether deletion of a further pathway included in the candidate model for reconstruction is possible, the determination being based on whether a remaining pathway has the number of channels equal to or less than the threshold;

if it is determined that deletion of the further pathway is possible, subject the candidate model for reconstruction to each of the training, the pruning, and the generating in an iterative manner to stabilize pruning results and progressively reduce the number of pathways until no further pathway deletion is possible; and

cause a display device to display a pruning history of the candidate model.

2 . The apparatus according to claim 1 , wherein if there is no pathway that is deletable among pathways included in the candidate model for reconstruction, or if a predetermined termination condition is satisfied, the processor selects the candidate model for reconstruction as a reconstructed model for which the pruning has been completed.

3 . The apparatus according to claim 1 , further comprising a storage,

wherein the processor is configured to:

perform the pruning on the neural network model or the candidate model for reconstruction multiple times, and

store, in the storage, the pruning history which associates a history of pruning results of the pruning performed multiple times and a history of the candidate model for reconstruction based on the respective pruning results.

4 . The apparatus according to claim 3 , wherein the processor is configured to reconstruct a candidate model for reconstruction based on the pruning history.

5 . The apparatus according to claim 3 , wherein the processor is configured to determine whether re-training is necessary or not based on the pruning history.

6 . The apparatus according to claim 3 , wherein the processor is configured to perform the pruning multiple times while changing a threshold relating to the pruning.

7 . The apparatus according to claim 3 , wherein the processor is further configured to cause the number of remaining parameters of each of the pathways to be displayed as the pruning history on a display.

8 . The apparatus according to claim 7 , wherein the processor is configured to cause (a) a pathway in which the number of remaining parameters is equal to or below a first threshold or (b) a pathway in which the number of remaining parameters is equal to or above a second threshold to be displayed in a highlighted manner, the second threshold being greater than the first threshold.

9 . The apparatus according to claim 1 , wherein the processor is configured to:

perform the pruning on the neural network model or the candidate model for reconstruction multiple times, and

if there are a plurality of candidate models for reconstruction differing from each other in deleted pathway, determine whether or not deletion of a further pathway is possible from each of the candidate models for reconstruction.

10 . The apparatus according to claim 1 , wherein the processor is configured to:

perform the pruning on the neural network model or the candidate model for reconstruction multiple times;

determine whether or not there is a negative correlation between a plurality of pathways included in the neural network model or between a plurality of pathways included in the candidate model for reconstruction, and

in the presence of the negative correlation, generate a plurality of candidate models for reconstruction differing from each other in deleted pathway among the plurality of pathways.

11 . The apparatus according to claim 1 , wherein the number of parameters is at least one of a number of weight coefficients or a number of channels.

12 . The apparatus according to claim 1 , wherein the processor is configured to train the neural network model and the candidate model for reconstruction in a training method that causes sparsification in units of channels.

13 . A learning method comprising:

training a neural network model having a plurality of pathways arranged between an input layer and an output layer, the neural network model including hidden layers with parameters comprising weight coefficients or channels and generating a trained model;

performing pruning on the trained model by calculating, for each pathway, an L2 norm of each channel in the hidden layer included in the pathway, and the number of channels having an L2 norm larger than a predetermined threshold;

generating a candidate model for reconstruction by deleting a pathway in which the number of channels having the L2 norm larger than the predetermined threshold is equal to or less than a threshold, and constructing the candidate model with the remaining pathways;

determining whether deletion of a further pathway included in the candidate model for reconstruction is possible or not, the determination being based on whether a remaining pathway has the number of channels equal to or less than the threshold;

if it is determined that deletion of the further pathway is possible, subjecting the candidate model for reconstruction to each of the training, the pruning, and the generating in an iterative manner to stabilize pruning results and progressively reduce the number of pathways until no further pathway deletion is possible; and

causing a display device to display a pruning history of the candidate model.