Processing method and accelerating device
The present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption.
1. A processing device, comprising:
a coarse-grained pruning circuit configured to perform coarse-grained pruning on weights of a neural network to obtain pruned weights, wherein the neural network includes a convolutional layer and a weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), Nfin represents a count of input feature maps, Nfout represents a count of output feature maps, (Kx, Ky) is a size of a convolution kernel, and the convolutional layer has Nfin*Nfout*Kx*Ky weights;
and an operation circuit configured to train the neural network according to the pruned weight;
wherein the coarse-grained pruning circuit is configured to:
select M weights from the weights of the neural network through a sliding window;
determine that the M weights meet a preset condition, wherein the preset condition is that an information quantity of the M weights is less than a first given threshold;
and based on a determination that the M weights meet the preset condition,
set at least a portion of the selected M weights to 0 to obtain a portion of the pruned weights,
wherein the coarse-grained pruning circuit is further configured to:
perform coarse-grained pruning on the weight of the convolutional layer, where the sliding window is a four-dimensional sliding window with a size of Bfin*Bfout*Bx*By, Bfin is a positive integer greater than 0 and less than or equal to Nfin, Bfout is a positive integer greater than 0 and less than or equal to Nfout, Bx is a positive integer greater than 0 and less than or equal to Kx, and By is a positive integer greater than 0 and less than or equal to Ky,
make the sliding window slide Sfin stride in the direction of Bfin, or slide Sfout stride in the direction of Bfout, or slide S stride in the direction of Bx, or slide Sy stride in the direction of By, where Sfin is a positive integer greater than 0 and less than or equal to Bfin, Sfout is a positive integer greater than 0 and less than or equal to Bfout, Sx is a positive integer greater than 0 and less than or equal to Bx, and Sy is a positive integer greater than 0 and less than or equal to By,
select M weights from the Nfin*Nfout*Kx*Ky weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights may be set to 0, where M=Bfin*Bfout*Bx*Bv.
2. The processing device of claim 1 , wherein
the information quantity of the M weights is an arithmetic mean of an absolute value of the M weights, a geometric mean of the absolute value of the M weights or a maximum value of the absolute value of the M weights; the first given threshold is a first threshold, a second threshold or a third threshold; and the information quantity of the M weights being less than the first given threshold includes:
the arithmetic mean of the absolute value of the M weights being less than the first threshold, or the geometric mean of the absolute value of the M weights being less than the second threshold, or the maximum value of the M weights being less than the third threshold.
3. The processing device of claim 1 , wherein the coarse-grained pruning circuit and the operation circuit are configured to repeat performing coarse-grained pruning on the weights of the neural network and training the neural network according to the pruned weights until no weight meets the preset condition without losing a preset precision.
4. The processing device of claim 1 ,
wherein the neural network includes a fully connected layer and a Long Short Term Memory (LSTM) layer, where a weight of the fully connected layer is a two-dimensional matrix (Nin, Nout), Nin represents a count of input neurons and Nout represents a count of output neurons, and the fully connected layer has Nin*Nout weights;
a weight of LSTM layer is composed of m weights of the fully connected layer, m is a positive integer greater than 0, and an ith weight of the fully connected layer is (Nin_i, Nout_i), where i is a positive integer greater than 0 and less than or equal to m, Nin_i represents a count of input neurons of the ith weight of the fully connected layer and Nout_i represents a count of output neurons of the ith weight of the fully connected layer;
the coarse-grained pruning unit is specifically configured to: perform coarse-grained pruning on the weight of the fully connected layer, where the sliding window is a sliding window with a size of Bin*Bout, Bin is a positive integer greater than 0 and less than or equal to Nin, and Bout is a positive integer greater than 0 and less than or equal to Nout,
make the sliding window slide Sin stride in a direction of Bin, or slide Sout stride in a direction of Bout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout,
select M weights from the Nin*Nout weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights may be set to 0, where M=Bin*Bout,
perform coarse-grained pruning on the weight of the LSTM layer, where the size of the sliding window is Bin_i*Bout_i, Bin_i is a positive integer greater than 0 and less than or equal to Nini, and Bout_i is a positive integer greater than 0 and less than or equal to Nout_i,
make the sliding window slide Sini stride in the direction of Bin_i, or slide Sout_i stride in the direction of Bout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i, and
select M weights from the Bin_i*Bouti weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights may be set to 0, where M=Bin_i*Bouti.
5. The processing device of claim 1 , wherein the operation circuit is specifically configured to:
retrain the neural network by a back-propagation algorithm according to the pruned weight.
6. The processing device of claim 1 , further comprising
a quantization circuit configured to quantize the weight of the neural network and/or perform a first operation on the weight of the neural network after the coarse-grained pruning circuit performs coarse-grained pruning on the weight of the neural network, and before the operation circuit retrains the neural network according to the pruned weight to reduce a count of weight bits of the neural network.
7. A neural network operation device, comprising: a first processing device; and one or more second processing devices communicatively connected to the first processing device via a Peripheral Component Interconnect Express (PCIE) bus to transfer data to support operations of a neural network, wherein the neural network includes a convolutional layer and a weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), Nfin represents a count of input feature maps, Nfout represents a count of output feature maps, (Kx, Ky) is a size of a convolution kernel, and the convolutional layer has Nfin*Nfout*Kx*Ky weights, wherein each of the first processing device and the one or more second processing devices includes:
a coarse-grained pruning circuit configured to perform coarse-grained pruning on weights of the neural network to obtain pruned weights;
and an operation circuit configured to train the neural network according to the pruned weight, wherein the coarse-grained pruning circuit is configured to:
select M weights from the weights of the neural network through a sliding window,
determine that the M weights meet a preset condition, wherein the preset condition is that an information quantity of the M weights is less than a first given threshold;
and set at least one of the selected M weights to zero to obtain a portion of the pruned weights based on a determination that the M weights meet the preset condition,
wherein the coarse-grained pruning circuit is further configured to:
perform coarse-grained pruning on the weight of the convolutional layer, where the sliding window is a four-dimensional sliding window with a size of Bfin*Bfout*Bx*By, Bfin is a positive integer greater than 0 and less than or equal to Nfin, Bfout is a positive integer greater than 0 and less than or equal to Nfout, Bx is a positive integer greater than 0 and less than or equal to Kx, and By is a positive integer greater than 0 and less than or equal to Ky,
make the sliding window slide Sfin stride in the direction of Bfin, or slide Sfout stride in the direction of Bfout, or slide S stride in the direction of Bx, or slide Sy stride in the direction of By, where Sfin is a positive integer greater than 0 and less than or equal to Bfin, Sfout is a positive integer greater than 0 and less than or equal to Bfout, Sx is a positive integer greater than 0 and less than or equal to Bx, and Sy is a positive integer greater than 0 and less than or equal to By,
select M weights from the Nfin*Nfout*Kx*Ky weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights set to 0, where M=Bfin*Bfout*Bx*Bv.
8. A processing method, comprising performing coarse-grained pruning on weights of a neural network to obtain pruned weights, wherein the neural network includes a convolutional layer and a weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), Nfin represents a count of input feature maps, Nfout represents a count of output feature maps, (Kx, Ky) is a size of a convolution kernel, and the convolutional layer has Nfin*Nfout*Kx*Ky weights; and
training the neural network according to the pruned weights;
wherein the performing coarse-grained pruning on the weights of the neural network to obtain the pruned weights includes:
selecting M weights from the weights of the neural network through a sliding window,
determining that the M weights meet a preset condition, wherein the preset condition is that an information quantity of the M weights is less than a first given threshold;
based on a determination that the M weights meet the preset condition,
setting at least a portion of the selected M weights to zero to obtain a portion of the pruned weights, and
wherein the performing coarse-grained pruning further includes:
performing coarse-grained pruning on the weight of the convolutional layer, the sliding window is a four-dimensional sliding window with a size of Bfin*Bfout*Bx*By, where Bfin is a positive integer greater than 0 and less than or equal to Nfin, Bfout is a positive integer greater than 0 and less than or equal to Nfout, Bx is a positive integer greater than 0 and less than or equal to Kx, and By is a positive integer greater than 0 and less than or equal to Ky,
making the sliding window slide Sfin stride in the direction of Bfin, or slide Sfout stride in the direction of Bfout, or slide S stride in the direction of Bx, or slide Sy stride in the direction of By, where Sfin is a positive integer greater than 0 and less than or equal to Bfin, Sfout is a positive integer greater than 0 and less than or equal to Bfout, Sx is a positive integer greater than 0 and less than or equal to Bx, and Sy is a positive integer greater than 0 and less than or equal to By,
selecting M weights from the Nfin*Nfout*Kx*Ky weights through the sliding window, and when the M weights meet the preset condition, setting all or part of the M weights to 0, where M=Bfin*Bfout*Bx*By.
9. The processing method of claim 8 , wherein the information quantity of the M weights is an arithmetic mean of an absolute value of the M weights, a geometric mean of the absolute value of the M weights or a maximum value of the absolute value of the M weights; the first given threshold is a first threshold, a second threshold or a third threshold; and the information quantity of the M weights being less than the first given threshold includes:
the arithmetic mean of the absolute value of the M weights being less than the first threshold, or the geometric mean of the absolute value of the M weights being less than the second threshold, or the maximum value of the M weights being less than the third threshold.
10. The processing method of claim 8 , further comprising
repeating performing coarse-grained pruning on the weights of the neural network and training the neural network according to the pruned weights until no weight meets the preset condition without losing a preset precision.
11. The processing method of claim 8 ,
wherein the neural network includes a fully connected layer and a Long Short Term Memory (LSTM) layer, where a weight of the fully connected layer is a two-dimensional matrix (Nin, Nout), Nin represents a count of input neurons and Nout represents a count of output neurons, and the fully connected layer has Nin*Nout weights;
a weight of LSTM layer is composed of m weights of the fully connected layer, m is a positive integer greater than 0, and an ith weight of the fully connected layer is (Nin_i, Nout_i), where i is a positive integer greater than 0 and less than or equal to m, Nin_i represents a count of input neurons of the ith weight of the fully connected layer and Nout_i represents a count of output neurons of the ith weight of the fully connected layer;
the coarse-grained pruning unit is specifically configured to:
performing coarse-grained pruning on the weight of the fully connected layer, where the sliding window is a sliding window with a size of Bin*Bout, Bin is a positive integer greater than 0 and less than or equal to Nin, and Bout is a positive integer greater than 0 and less than or equal to Nout,
make the sliding window slide Sin stride in a direction of Bin, or slide Sout stride in a direction of Bout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout,
select M weights from the Nin*Nout weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights set to 0, where M=Bin*Bout,
perform coarse-grained pruning on the weight of the LSTM layer, where the size of the sliding window is Bin_i*Bout_i, Bin_i is a positive integer greater than 0 and less than or equal to Nini, and Bout_i is a positive integer greater than 0 and less than or equal to Nout_i,
make the sliding window slide Sini stride in the direction of Bin_i, or slide Sout_i stride in the direction of Bout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i, and
select M weights from the Bin_i*Bouti weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights may be set to 0, where M=Bin_i*Bouti.
12. The processing method of claim 8 , wherein the training the neural network according to the pruned weight is:
retraining the neural network by a back-propagation algorithm according to the pruned weight.
13. The processing method of claim 8 , wherein after performing coarse-grained pruning on the weight of the neural network, and before retraining the neural network, the method further includes:
quantizing the weight of the neural network and/or performing a first operation on the weight of the neural network to reduce a count of weight bits of the neural network.