Method and device for generating neuron network compensated for loss due to pruning

1 . A computer-implemented method for generating a neural network that compensates for information loss due to pruning performed by a processor, the method comprising:

obtaining a trained neural network;

pruning at least one neuron in the trained neural network; and

updating one or more parameter values of a next layer in a pruned neural network based on one or more parameter values of at least one neuron among remaining neurons in a pruned target layer having the pruned neuron,

wherein the step of updating includes the steps of:

selecting another neuron or a combination of other neurons most similar to the pruned neuron among the remaining neurons in the target layer;

computing a merging value based on one or more parameter values of the selected other neuron or one or more parameter values of a plurality of other neurons underlying the combination and one or more parameter values of the pruned neuron; and

computing a merging matrix including the merging value based on one or more parameter values of remaining neurons in the pruned target layer and one or more parameter values of a neuron in an unpruned target layer,

wherein one or more parameter values of neuron in the next layer in the pruned neural network is updated through the following equation,

W

i

+

1

′

=

Z

i

⁢

W

i

+

1

[

Equation

]

wherein W′ i+1 is a matrix of the parameter value of neuron in the next layer in the updated, the pruned neural network, Z i is the merging matrix, and W i+1 is a matrix of one or more parameter values of neuron in the next layer in the un-updated, the pruned neural network,

wherein the pruning comprises structured channel pruning of a convolutional neural network (CNN), and the compensation is applied to the filter weights of a next convolutional layer in the CNN, wherein the merging matrix Zi is configured to recover inference performance degradation caused by the pruning.

2 . The method according to claim 1 , wherein the step of updating further includes the step of updating one or more parameter values of a neuron in the next layer in the pruned neural network based on one or more parameter values of the merging matrix and one or more parameter values of a neuron in the next layer in an unpruned neural network.

3 . The method according to claim 1 , wherein the other neuron most similar to the pruned neuron is a neuron having a most similar orientation among the remaining neurons in the pruned target layer,

the merging value is a ratio between the parameter value of the selected neuron and the parameter value of the pruned neuron.

4 . The method according to claim 1 , wherein the combination of other neurons most similar to the pruned neuron is one in which a result of combining two or more of the remaining neurons in the pruned target layer has the most similar orientation to the pruned neuron, and the combination is a sum of the one or more parameter values.

5 . The method according to claim 4 , wherein the merging matrix includes a plurality of merging values, and each of the plurality of merging values is a coefficient of a sum of one or more parameter values for respective sub neurons of the selected combination.

6 . The method according to claim 1 , wherein the merging matrix is computed by decomposing a matrix of one or more parameter values of neurons in the unpruned target layer into a matrix of one or more parameter values of the remaining neurons in the pruned target layer and the merging matrix through a matrix decomposition way.

7 . The method according to claim 1 , wherein the neural network includes at least some of a plurality of fully connected layers and a plurality of convolutional layers,

the fully connected layer includes a node as the neuron, and the convolutional layer includes a filter as the neuron,

the parameter of the neuron includes at least one of a node parameter and a filter parameter.

8 . The method according to claim 1 , wherein the neural network includes an activation function between the target layer and the next layer.

9 . The method according to claim 8 , wherein when the activation function is ReLU, the step of updating is performed when the merging matrix satisfies a preset specific condition,

the specific condition includes that the merging matrix Zi has only non-negative component value.

10 . The method according to claim 9 , wherein the specific condition further includes that the merging matrix has at most one positive component value per column.

11 . A non-transitory computer-readable recording medium which records a program for executing the method according to claim 1 .