Method and device for compressing neural network

1 . A method for compressing a neural network, comprising:

obtaining an original neural network to be compressed, the original neural network comprising a plurality of parameters to be compressed;

dividing the plurality of parameters of the original neural network into J blocks to be compressed, where J is an integer greater than 1;

compressing a j th block of the J blocks with K j different compression ratios to generate K j operation branches, where j and K j are integers, 1≤j≤J, and K j ≥1;

obtaining, for the j th block, a set of K j weighting factors corresponding to the respective K j operation branches, where the K j weighting factors have respective initial values;

replacing the j th block with the K j operation branches weighted by the set of K j weighting factors, to generate a replacement neural network;

performing forward propagation to the replacement neural network based on a preset dataset to generate K j operation results, where a weighted sum operation is performed on the K j operation results with the K j weighting factors and a result of the weighted sum operation is used as an output of the j th block, and where the K j operation results is generated by:

compressing an input data from the preset dataset to generate K j compressed input data corresponding to the K j operation branches respectively, where the K j compressed input data have K j different accuracies corresponding to the K j different compression ratios of the K j operation branches respectively; and

performing the forward propagation on the K j replaced operation branches and the K j compressed input data to generate the K j operation results;

performing backward propagation to the replacement neural network based on the preset dataset, where, for the j th block, updated values of the K j weighting factors are calculated based on a model loss of the replacement neural network relative to the original neural network, wherein the model loss is determined based on a sum of a loss function determined based on an application type of the original neural network, and a performance index related to a hardware index of a hardware platform on which the original neural network is to be deployed; and

determining, for the j th block, an operation branch corresponding to a maximum value of the updated values of the K j weighting factors as a compressed j th block.

2 . The method of claim 1 , wherein the original neural network comprising one or more operation layers to be compressed and each operation layer comprising one or more operation channels to be compressed, and dividing the plurality of parameters of the original neural network into J blocks to be compressed comprises: dividing parameters in the same operation layer into a respective block to be compressed.

3 . The method of claim 2 , wherein compressing a j th block of the J blocks with K j different compression ratios comprises:

calculating L1 norms of all operations channels of the j th block, respectively;

selecting P 1 , P 2 , . . . , P Kj operations channels with the smallest L1 norms from all operations channels of the j th block based on the K j compression ratios of the j th block, where P 1 , P 2 , . . . , P Kj are integers greater than or equal to 1, and less than J; and

setting values of all parameters in the P 1 , P 2 , . . . , P Kj operations channels of the j th block to zero to generate the K j operation branches, respectively.

4 . The method of claim 1 , wherein the original neural network comprising one or more operation layers to be compressed, each operation layer comprising one or more operation channels to be compressed and each operation channel comprising one or more operation kernels to be compressed, and dividing the plurality of parameters of the original neural network into J blocks to be compressed comprises: dividing parameters in the same operation channel into a respective block to be compressed.

5 . The method of claim 4 , wherein compressing a j th block of the J blocks with K j different compression ratios comprises:

calculating L1 norms of all operations kernels of the j th block, respectively;

selecting P 1 , P 2 , . . . , P Kj operations kernels with the smallest L1 norms from all operations kernels of the j th block based on the K j compression ratios of the j th block, where P 1 , P 2 , . . . , P Kj are integers greater than or equal to 1, and less than J; and

setting values of all parameters in the P 1 , P 2 , . . . , P Kj operations kernels of the j th block to zero to generate the K j operation branches, respectively.

6 . The method of claim 1 , wherein the original neural network comprising one or more operation layers to be compressed, each operation layer comprising one or more operation channels to be compressed, each operation channel comprising one or more operation kernels to be compressed, and each operation kernel comprising one or more parameters to be compressed, and dividing the plurality of parameters of the original neural network into J blocks to be compressed comprises: dividing parameters in the same operation kernel into a respective block to be compressed.

7 . The method of claim 6 , wherein compressing a j th block of the J blocks with K j different compression ratios comprises:

calculating absolute values of all parameters of the j th block, respectively;

selecting P 1 , P 2 , . . . , P Kj parameters with the smallest absolute values from all parameters of the j th block based on the K j compression ratios of the j th block, where P 1 , P 2 , . . . , P Kj are integers greater than or equal to 1, and less than J; and

setting values of the P 1 , P 2 , . . . , P Kj parameters of the j th block to zero to generate the K j operation branches, respectively.

8 . The method of claim 1 , wherein the K j compression ratios of the j th block have respective initial values R j , R j +Δ, R j +2Δ, . . . , R j +(K j −1)Δ, where 0<R j <1, 0<Δ<1, and 0<(R j +(K j −1)Δ)<1.

9 . The method of claim 8 , further comprising:

performing the forward propagation and the backward propagation to the replacement neural network iteratively for multiple times based on the preset dataset;

wherein in the iteration process of the forward propagation and the backward propagation, the updated values of the K j weighting factors obtained in a backward propagation serve as weighting factors to be used in a forward propagation next to the backward propagation.

10 . The method of claim 9 , further comprising:

after the forward propagation and the backward propagation to the replacement neural network are iteratively performed for M times based on the preset dataset, obtaining updated values of the K j compression ratios based on the updated values of the K j weighting factors.

11 . The method of claim 10 , wherein obtaining updated values of the K j compression ratios based on the updated values of the K j weighting factors comprises:

setting the updated values of the K j compression ratio to be smaller than their respective initial values, when an updated value of the weighting factor corresponding to a first operation branch of the K j operation branches is greater than or equal to a preset weighting factor threshold;

setting the updated values of the K j compression ratio to be greater than their respective initial values, when an updated value of the weighting factor corresponding to a

K

j

th

operation branch of the K j operation branches is greater than or equal to the preset weighting factor threshold; and

setting the updated values of the K j compression ratio to be equal to their respective initial values, when the updated values of the weighting factors corresponding to the first operation branch and the

K

j

th

operation branch of the K j operation branches are smaller than the preset weighting factor threshold.

12 . The method of claim 9 , further comprising:

calculating, in the iteration process, a model size of the replacement neural network based on the operation branch corresponding to the maximum value of the updated values of the K j weighting factors;

obtaining a change in the model size of the replacement neural network calculated after each iteration of the iteration process; and

stopping the iteration process when the change is within a preset range.

13 . The method of claim 1 , wherein the application type of the original neural network comprises: classification, positioning, detection or segmentation.

14 . The method of claim 1 , wherein the one or more operation layers to be compressed comprises a convolutional layer, an activation layer, a batch normalization layer, a pooling layer, or a fully connected layer.

15 . The method of claim 1 , further comprising:

normalizing values of the K j weighting factors before performing the weighted sum operation on the K j operation results generated by the K j operation branches with the K j weighting factors.

16 . The method of claim 1 , wherein the hardware index of a hardware platform on which the original neural network is to be deployed comprises: a storage space, a number of floating-point operations, a delay time or a power consumption.

17 . A device for compressing a neural network, comprising:

a processor; and

a memory, wherein the memory stores program instructions that are executable by the processor, and when executed by the processor, the program instructions cause the processor to perform:

obtaining an original neural network to be compressed, the original neural network comprising a plurality of parameters to be compressed;

dividing the plurality of parameters of the original neural network into J blocks to be compressed, where J is an integer greater than 1;

compressing a j th block of the J blocks with K j different compression ratios to generate K j operation branches, where j and K j are integers, 1≤j≤J, and K j ≥1;

obtaining, for the j th block, a set of K j weighting factors corresponding to the respective K j operation branches, where the K j weighting factors have respective initial values;

replacing the j th block with the K j operation branches weighted by the set of K j weighting factors, to generate a replacement neural network;

performing forward propagation to the replacement neural network based on a preset dataset to generate K j operation results, where a weighted sum operation is performed on the K j operation results with the K j weighting factors and a result of the weighted sum operation is used as an output of the j th block, and where the K j operation results is generated by:

compressing an input data from the preset dataset to generate K j compressed input data corresponding to the K j operation branches respectively, where the K j compressed input data have K j different accuracies corresponding to the K j different compression ratios of the K j operation branches respectively; and

performing the forward propagation on the K j replaced operation branches and the A compressed input data to generate the K j operation results:

performing backward propagation to the replacement neural network based on the preset dataset, where, for the j th block, updated values of the K j weighting factors are calculated based on a model loss of the replacement neural network relative to the original neural network, wherein the model loss is determined based on a sum of a loss function determined based on an application type of the original neural network, and a performance index related to a hardware index of a hardware platform on which the original neural network is to be deployed; and

determining, for the j th block, an operation branch corresponding to a maximum value of the updated values of the K j weighting factors as a compressed j th block.

18 . A non-transitory computer-readable storage medium having stored therein instructions that, when executed by a processor, cause the processor to perform a method for compressing a neural network, the method comprising:

obtaining an original neural network to be compressed, the original neural network comprising a plurality of parameters to be compressed;

dividing the plurality of parameters of the original neural network into J blocks to be compressed, where J is an integer greater than 1;

compressing a j th block of the J blocks with K j different compression ratios to generate K j operation branches, where j and K j are integers, 1≤j≤J, and K j ≥1;

obtaining, for the j th block, a set of K j weighting factors corresponding to the respective K j operation branches, where the K j weighting factors have respective initial values;

replacing the j th block with the K j operation branches weighted by the set of K j weighting factors, to generate a replacement neural network;

performing forward propagation to the replacement neural network based on a preset dataset to generate K j operation results, where a weighted sum operation is performed on the K j operation results with the K j weighting factors and a result of the weighted sum operation is used as an output of the j th block, and where the K j operation results is generated by:

compressing an input data from the preset dataset to generate K j compressed input data corresponding to the K j operation branches respectively, where the K j compressed input data have K j different accuracies corresponding to the K j different compression ratios of the K j operation branches respectively; and

performing the forward propagation on the K j replaced operation branches and the K j compressed input data to generate the K j operation results;

performing backward propagation to the replacement neural network based on the preset dataset, where, for the j th block, updated values of the K j weighting factors are calculated based on a model loss of the replacement neural network relative to the original neural network, wherein the model loss is determined based on a sum of a loss function determined based on an application type of the original neural network, and a performance index related to a hardware index of a hardware platform on which the original neural network is to be deployed; and

determining, for the j th block, an operation branch corresponding to a maximum value of the updated values of the K j weighting factors as a compressed j th block.