Processing method and accelerating device
The present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption.
1. A data compression method, comprising: performing coarse-grained pruning on weights of a neural network, which includes: selecting one or more weights of the weights from the neural network through a sliding window, and setting all or part of the one or more weights to 0 when the one or more weights meet a preset condition, wherein the preset condition is a condition in which information quantity of the one or more weights is less than a first given threshold, and the information quantity is an arithmetic mean of an absolute value of the one or more weights, a geometric mean of the absolute value of the one or more weights, or a maximum value of the one or more weights; performing a first retraining on the neural network, where the one or more weights which have been set to 0 remain 0 in the first retraining; repeating selecting the one or more weights from the neural network through the sliding window; setting all or part of the one or more weights to 0 when the one or more weights meet the preset condition; and performing the first retraining on the neural network until no weight of the one or more weights is set to 0 without losing a preset precision; quantizing the weights of the neural network, which includes: grouping the weights of the neural network, performing a clustering operation on each group of the weights by using a clustering algorithm to generate multiple classes of the weights, computing a center weight of each class of the multiple classes, and replacing all the weights in each class of the multiple classes by the center weights; encoding the center weights and center weights codes to generate a weight codebook, wherein the weight codebook includes the center weights, the center weights codes, and a correspondence between the center weights and the respective center weight codes; and encoding the center weight codes to generate a weight dictionary.
2. The data compression method of claim 1 , wherein, after the encoding of the center weights, the method further includes:
performing a second retraining on the neural network, wherein,
only the weight codebook is trained during the second retraining of the neural network, and
the weight dictionary remains unchanged.
3. The data compression method of claim 1 , wherein
the first given threshold is a first threshold, a second threshold, or a third threshold; and
the information quantity of the one or more weights being less than the first given threshold includes:
the arithmetic mean of the absolute value of the one or more weights being less than the first threshold, or the geometric mean of the absolute value of the one or more weights being less than the second threshold, or the maximum value of the one or more weights being less than the third threshold.
4. The data compression method of claim 1 , wherein performing the coarse-grained pruning on the weights of a fully connected layer of the neural network includes: setting the weights of the fully connected layer being a two-dimensional matrix (Nin, Nout), where Nin represents a count of input neurons and Nout represents a count of output neurons, and the fully connected layer has Nin*Nout weights; setting a size of the sliding window being Bin*Bout, where Bin is a positive integer greater than 0 and less than or equal to Nin, and Bout is a positive integer greater than 0 and less than or equal to Nout; making the sliding window slide Sin stride in a direction of Bin, or slide Sout stride in a direction of Bout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout; and selecting the one or more weights from Nin*Nout weights through the sliding window, and when the one or more weights meet the preset condition, all or part of the one or more weights are set to 0, where M=Bin*Bout.
5. The data compression method of claim 1 , wherein performing the coarse-grained pruning on the weights of a convolutional layer of the neural network includes: setting the weights of the convolutional layer of the neural network being a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin represents a count of input feature maps, Nfout represents a count of output feature maps, (Kx, Ky) is a size of a convolution kernel, and the convolutional layer has Nfin*Nfout*Kx*Ky weights; setting the sliding window being a four-dimensional sliding window with a size of Bfin*Bfout*Bx*By, where Bfin is a positive integer greater than 0 and less than or equal to Nfin, Bfout is a positive integer greater than 0 and less than or equal to Nfout, Bx is a positive integer greater than 0 and less than or equal to Kx, and By is a positive integer greater than 0 and less than or equal to Ky; making the sliding window slide Sfin stride in a direction of Bfin, or slide Sfout stride in a direction of Bfout, or slide S stride in a direction of Bx, or slide Sy stride in a direction of By, where Sfin is a positive integer greater than 0 and less than or equal to Bfin, Sfout is a positive integer greater than 0 and less than or equal to Bfout, Sx is a positive integer greater than 0 and less than or equal to Bx, and Sy is a positive integer greater than 0 and less than or equal to By; and selecting the one or more weights from Nfin*Nfout*Kx*Ky weights through the sliding window, and when the one or more weights meet the preset condition, all or part of the one or more weights are set to 0, where M=Bfin*Bfout*Bx*By.
6. The data compression method of claim 1 , wherein performing the coarse-grained pruning on the weights of a LSTM layer of the neural network includes: setting the weights of the LSTM layer being composed of a plurality weights of a fully connected layer, and an i th weight of the fully connected layer is a two-dimensional matrix (Nin_i, Nout_ 1 ), where 1 is a positive integer greater than 0 and less than or equal to m, Nin_i represents a count of input neurons of the i weight of the fully connected layer, and Nout_i represents a count of output neurons of the i weight of the fully connected layer; setting a size of the sliding window being Bin_i*Bout_ 1 , where Bin_ 1 is a positive integer greater than 0 and less than or equal to Nin_i, and Bout_i is a positive integer greater than 0 and less than or equal to Nout_ 1 ; making the sliding window slide Sin_i stride in a direction of Bin_ 1 , or slide Sout_ 1 stride in a direction of Bout_i 1 , where Sin_ 1 is a positive integer greater than 0 and less than or equal to Bin i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i; and selecting one or more weights from Bin i*Bout_i weights through the sliding window, and when the one or more weights meet the preset condition, all or part of the one or more weights are set to 0, where M=Bin_i*Bout_ 1 .
7. The data compression method of claim 1 , wherein the first retraining adopts a back propagation algorithm, and the one or more weights which have been set to 0 remain 0 in the first retraining.
8. The data compression method of claim 1 ,
wherein the grouping of the weights of the neural network includes grouping into a group, layer-type-based grouping, inter-layer-based grouping, and/or intra-layer-based grouping,
wherein the grouping into a group includes grouping all the weights of the neural network into a group,
grouping the weights of the neural network according to the layer-type-based grouping method includes:
grouping the weights of all convolutional layers, the weights of all fully connected layers, and the weights of all LSTM layers in the neural network into one group respectively,
wherein the grouping of the weights of the neural network by the inter-layer-based grouping method includes:
grouping the weights of one or a plurality of convolutional layers, one or a plurality of fully connected layers and one or a plurality of LSTM layers in the neural network into one group respectively,
wherein the grouping of the weights of the neural network by the intra-layer-based grouping method includes:
segmenting the weights in one layer of the neural network, where each segmented part forms a group.
9. The data compression method of claim 1 , wherein the clustering algorithm includes K-means, K-medoids, Clara, and/or Clarans.
10. The data compression method of claim 1 , wherein a center weight selection method of a class is: minimizing a cost function J(w,w 0 ).
11. The data compression method of claim 10 , wherein the cost function meets a condition:
J
(
w
,
w
0
)
=
∑
i
=
1
n
(
w
i
-
w
0
)
2
where w is all weights of a class, w0 is a center weight of the class, n is a count of weights in the class, wi is an i th weight of the class, and i is a positive integer greater than 0 and less than or equal to n.
12. The data compression method of claim 2 , wherein the second retraining performed on the neural network after the clustering and the encoding includes:
performing the second retraining on the neural network after the clustering, and the encoding by using a back propagation algorithm, where the one or more weights that have been set to 0 in the second retraining remains 0 all time, and only the weight codebook is retrained, the weight dictionary is not retrained.
13. A data compression device, comprising: a memory configured to store an operation instruction; and a processor configured to: perform coarse-grained pruning on weights of a neural network, which includes: select one or more weights of the weights from the neural network through a sliding window, and set all or part of the one or more weights to 0 when the one or more weights meet a preset condition, wherein the preset condition is a condition in which information quantity of the one or more weights is less than a first given threshold, and the information quantity is an arithmetic mean of an absolute value of the one or more weights, a geometric mean of the absolute value of the one or more weights, or a maximum value of the one or more weights; perform a first retraining on the neural network, where the one or more weights which have been set to 0 remain 0 in the first retraining; repeat selection of the one or more weights from the neural network through the sliding window; set all or part of the one or more weights to 0 when the one or more weights meet the preset condition; and perform the first retraining on the neural network until no weight of the one or more weights is set to 0 without losing a preset precision; quantize the weights of the neural network, wherein the processor is further configured to: group the weights of the neural network, perform a clustering operation on each group of the weights by using a clustering algorithm to generate multiple classes of the weights, compute a center weight of each class of the multiple classes, and replace all the weights in each class of the multiple classes by the center weights; encode the center weights and center weights codes to generate a weight codebook, wherein the weight codebook includes the center weights, the center weights codes, and a correspondence between the center weights and the respective center weight codes; and encode the center weight codes to generate a weight dictionary.
14. The data compression device of claim 13 , the processor is further configured to perform a second retraining on the neural network, wherein, only the weight codebook is trained during the second retraining of the neural network, and the weight dictionary remains unchanged.
15. The data compression device of claim 13 , wherein
the first given threshold is a first threshold, a second threshold, or a third threshold; and
the information quantity of the one or more weights being less than the first given threshold includes:
the arithmetic mean of the absolute value of the one or more weights being less than the first threshold, or the geometric mean of the absolute value of the one or more weights being less than the second threshold, or the maximum value of the one or more weights being less than the third threshold.
16. An electronic device, comprising: a data compression device that includes: a memory configured to store an operation instruction; and a processor configured to: perform coarse-grained pruning on weights of a neural network, which includes: select one or more weights of the weights from the neural network through a sliding window, and set all or part of the one or more weights to 0 when the one or more weights meet a preset condition, wherein the preset condition is a condition in which information quantity of the one or more weights is less than a first given threshold, and the information quantity is an arithmetic mean of an absolute value of the one or more weights, a geometric mean of the absolute value of the one or more weights, or a maximum value of the one or more weights; perform a first retraining on the neural network, where the one or more weights which have been set to 0 remains 0 in the first retraining; repeat selection of the one or more weights from the neural network through the sliding window; set all or part of the one or more weights to 0 when the one or more weights meet the preset condition; and perform the first retraining on the neural network until no weight of the one or more weights is set to 0 without losing a preset precision; quantize the weights of the neural network, wherein the processor is further configured to: group the weights of the neural network, perform a clustering operation on each group of the weights by using a clustering algorithm to generate multiple classes of the weights, compute a center weight of each class of the multiple classes, and replace all the weights in each class of the multiple classes by the center weights; encode the center weights and center weights codes to generate a weight codebook, wherein the weight codebook includes the center weights, the center weights codes, and a correspondence between the center weights and the respective center weight codes; and encode the center weight codes to generate a weight dictionary.