Neural network quantization method, device and related products
The invention provides a neural network quantization method and device and a related product. The neural network quantization method is used for quantizing data of a computation layer of a neural network. The technical scheme provided by the invention has the advantage of low cost.
1. A neural network quantization method, comprising:
obtaining a weight and input data of a target quantization layer of an original neural network, wherein the target quantization layer includes at least one computation layer of the original neural network;
determining a quantization parameter of a weight of a corresponding layer by using the weight of the target quantization layer of the original neural network; determining a quantization parameter of input data of a corresponding layer by using the input data of the target quantization layer of the original neural network, wherein both the weight and the input data of the target quantization layer follow a principle of not distorting a maximum absolute value; and
quantizing the target quantization layer of the original neural network according to the quantization parameter of the weight and the quantization parameter of the input data to generate a quantized weight and quantized input data.
2. The neural network quantization method of claim 1 , wherein the computation layer includes at least one of a convolution layer, a fully connected layer, an LRN layer, a deconvolution layer, a Reorg layer, and a Normalize layer.
3. The neural network quantization method of claim 1 , wherein the determining the quantization parameter of the weight of the corresponding layer by using the weight of the target quantization layer of the original neural network includes:
obtaining a maximum absolute value of a weight of each target quantization layer, and
determining a first quantization parameter and a second quantization parameter of the weight of the corresponding layer according to the maximum absolute value of the weight of each target quantization layer.
4. The neural network quantization method of claim 1 , wherein the determining the quantization parameter of the input data of the corresponding layer by using the input data of the target quantization layer of the original neural network includes:
obtaining a maximum absolute value of input data of each target quantization layer, and
determining a first quantization parameter and a second quantization parameter of input data of the corresponding layer according to the maximum absolute value of the input data of each target quantization layer.
5. The neural network quantization method of claim 1 , further comprising:
processing each target quantization layer of the original neural network by using a first quantization method, a second quantization method, or a third quantization method, wherein:
the first quantization method includes:
quantizing the weight of the corresponding layer by using a first quantization parameter of the weight of each target quantization layer to obtain a weight quantization result of the corresponding layer, and
quantizing the input data of the corresponding layer by using a first quantization parameter of the input data of each target quantization layer to obtain an input data quantization result of the corresponding layer,
the second quantization method includes:
obtaining a weight quantization intermediate parameter of the corresponding layer by using the first quantization parameter and a second quantization parameter of the weight of each target quantization layer,
obtaining the weight quantization result of the corresponding layer according to the weight quantization intermediate parameter,
obtaining a quantization intermediate parameter of the input data of the corresponding layer by using the first quantization parameter and a second quantization parameter of the input data of each target quantization layer, and
obtaining the input data quantization result of the corresponding layer according to the quantization intermediate parameter of the input data,
the third quantization method includes:
obtaining the weight quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the weight of each target quantization layer, and
obtaining the input data quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the input data of each target quantization layer.
6. The neural network quantization method of claim 1 , further comprising:
obtaining a weight quantization intermediate parameter of a corresponding channel by using a first weight quantization parameter and a second weight quantization parameter of each channel of each target quantization layer, wherein the target quantization layer includes a convolution layer and/or a fully connected layer,
obtaining a weight quantization result of the corresponding channel by using the weight quantization intermediate parameter of each channel, wherein the weight quantization result of each channel of each target quantization layer constitutes a weight quantization result of the corresponding layer,
obtaining a quantization intermediate parameter of the input data of the corresponding layer by using a first input data quantization parameter and a second input data quantization parameter of each target quantization layer, and
obtaining an input data quantization result of the corresponding layer by using the quantization intermediate parameter of the input data of each target quantization layer.
7. The neural network quantization method of claim 6 , further comprising:
processing each target quantization layer of the original neural network by using the first quantization method, the second quantization method, or the third quantization method, wherein the target quantization layer further includes at least one layer other than the convolution layer and/or the fully connected layer in the computation layers of the original neural network,
the first quantization method includes:
quantizing the weight of the corresponding layer by using the first quantization parameter of the weight of each target quantization layer to obtain the weight quantization result of the corresponding layer, and
quantizing the input data of the corresponding layer by using the first quantization parameter of the input data of each target quantization layer to obtain the quantization result of the input data of the corresponding layer,
the second quantization method includes:
obtaining a weight quantization intermediate parameter of the corresponding layer by using the first quantization parameter and the second quantization parameter of the weight of each target quantization layer,
obtaining a weight quantization result of the corresponding layer according to the weight quantization intermediate parameter,
obtaining the quantization intermediate parameter of the input data of the corresponding layer by using the first quantization parameter and the second quantization parameter of the input data of each target quantization layer, and
obtaining the input data quantization result of the corresponding layer according to the quantization intermediate parameter of the input data,
the third quantization method includes:
obtaining the weight quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the weight of each target quantization layer, and
obtaining the input data quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the input data of each target quantization layer.
8. A neural network quantization device, comprising:
a data reading unit configured to obtain a weight and input data of a target quantization layer of an original neural network, wherein the target quantization layer is at least one computation layer of the original neural network;
a quantization parameter determining unit configured to determine a quantization parameter of a weight of a corresponding layer by using the weight of the target quantization layer of the original neural network, and determine a quantization parameter of input data of the corresponding layer by using the input data of the target quantization layer of the original neural network, wherein both the weight and the input data of the target quantization layer follow a principle of not distorting a maximum absolute value; and
a quantization unit configured to quantize the target quantization layer of the original neural network according to the quantization parameter of the weight and the quantization parameter of the input data to generate a quantized weight and quantized input data.
9. The neural network quantization device of claim 8 , wherein the computation layer includes at least one of a convolution layer, a fully connected layer, an LRN layer, a deconvolution layer, a Reorg layer, and a Normalize layer.
10. The neural network quantization device of claim 8 , wherein:
the quantization parameter determining unit is configured to obtain a maximum absolute value of a weight of each target quantization layer, and determine a first quantization parameter and a second quantization parameter of the weight of the corresponding layer according to the maximum absolute value of the weight of each target quantization layer.
11. The neural network quantization device of claim 8 , wherein:
the quantization parameter determining unit is configured to obtain a maximum absolute value of input data of each target quantization layer, and determine a first quantization parameter and a second quantization parameter of the input data of the corresponding layer according to the maximum absolute value of the input data of each target quantization layer.
12. The neural network quantization device of claim 8 , further comprising:
a processing unit configured to process each target quantization layer of the original neural network by using a first quantization method, a second quantization method, or a third quantization method, wherein
the first quantization method includes:
quantizing the weight of the corresponding layer by using a first quantization parameter of the weight of each target quantization layer to obtain a weight quantization result of the corresponding layer, and
quantizing the input data of the corresponding layer by using a first quantization parameter of the input data of each target quantization layer to obtain a quantization result of the input data of the corresponding layer,
the second quantization method includes:
obtaining a weight quantization intermediate parameter of the corresponding layer by using the first quantization parameter and a second quantization parameter of the weight of each target quantization layer,
obtaining the weight quantization result of the corresponding layer according to the weight quantization intermediate parameter,
obtaining a quantization intermediate parameter of the input data of the corresponding layer by using the first quantization parameter and a second quantization parameter of the input data of each target quantization layer, and
obtaining an input data quantization result of the corresponding layer according to the quantization intermediate parameter of the input data,
the third quantization method includes:
obtaining the weight quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the weight of each target quantization layer, and
obtaining the input data quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the input data of each target quantization layer.
13. The neural network quantization device of claim 8 , further comprising:
a processing unit configured to obtain a weight quantization intermediate parameter of the corresponding channel by using a first weight quantization parameter and a second weight quantization parameter of each channel of each target quantization layer, wherein the target quantization layer includes a convolutional layer and/or a fully connected layer,
obtaining a weight quantization result of the corresponding channel by using the weight quantization intermediate parameter of each channel, wherein the weight quantization result of each channel of each target quantization layer constitutes a weight quantization result of the corresponding layer,
obtaining a quantization intermediate parameter of the input data of the corresponding layer by using a first input data quantization parameter and a second input data quantization parameter of each target quantization layer; and
obtaining an input data quantization result of the corresponding layer by using the quantization intermediate parameter of the input data of each target quantization layer.
14. The neural network quantization device of claim 13 , wherein:
the processing unit is further configured to process each target quantization layer of the original neural network by using a first quantization method, a second quantization method, or a third quantization method, wherein the target quantization layer further includes at least one layer other than the convolution layer and/or the fully connected layer in the computation layers of the original neural network, wherein:
the first quantization method includes:
quantizing the weight of the corresponding layer by using the first quantization parameter of the weight of each target quantization layer to obtain the weight quantization result of the corresponding layer, and
quantizing the input data of the corresponding layer by using the first quantization parameter of the input data of each target quantization layer to obtain the quantization result of the input data of the corresponding layer,
the second quantization method includes:
obtaining a weight quantization intermediate parameter of the corresponding layer by using the first quantization parameter and the second quantization parameter of the weight of each target quantization layer,
obtaining the weight quantization result of the corresponding layer according to the weight quantization intermediate parameter,
obtaining the quantization intermediate parameter of the input data of the corresponding layer by using the first quantization parameter and the second quantization parameter of the input data of each target quantization layer, and
obtaining an input data quantization result of the corresponding layer according to the quantization intermediate parameter of the input data,
the third quantization method includes:
obtaining the weight quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the weight of each target quantization layer, and
obtaining the input data quantization result of the corresponding layer by using the first quantization parameter and the second quantization parameter of the input data of each target quantization layer.