Hardware environment-based data quantization method and apparatus, and readable storage medium
A hardware environment-based data quantization method includes: parsing a model file under a current deep learning framework to obtain intermediate computational graph data and weight data that are independent of a hardware environment; performing calculation on image data in an input data set through a process indicated by an intermediate computational graph to obtain feature map data; separately performing uniform quantization on the weight data and the feature map data of each layer according to a preset linear quantization method, and calculating a weight quantization factor and a feature map quantization factor (S 103 ); combining the weight quantization factor and the feature map quantization factor to obtain a quantization parameter that makes hardware use shift instead of division; and finally, writing the quantization parameter and the quantized weight data to a bin file according to a hardware requirement so as to generate quantized file data (S 105 ).
1. A data quantization method, comprising:
performing parsing according to a model file under a current deep learning framework to obtain intermediate computational graph data and weight data that are independent of a hardware environment;
performing, based on the intermediate computational graph data and the weight data, calculation on image data in an input data set through a process indicated by an intermediate computational graph to obtain feature map data;
separately performing uniform quantization on the weight data and the feature map data of each layer according to a preset linear quantization method, and calculating a weight quantization factor and a feature map quantization factor;
combining the weight quantization factor and the feature map quantization factor to obtain a quantization parameter, wherein the quantization parameter is a parameter that makes hardware use shift instead of division; and
writing the quantization parameter and quantized weight data to a bin file according to a hardware requirement so as to generate quantized file data.
2. The data quantization method according to claim 1 , wherein before the step of writing the quantization parameter and the quantized weight data to the bin file according to the hardware requirement, the method further comprises:
reordering the quantization parameter and the quantized weight data such that a data format of the quantization parameter and the quantized weight data is a 64-channel parallel format.
3. The data quantization method according to claim 2 , wherein the step of performing parsing to obtain the intermediate computational graph data and the weight data of the current deep learning framework comprises:
parsing the model file by use of a Neural Network Virtual Machine (NNVM) component in an NNVM compiler to obtain the intermediate computational graph data; and
by use of a Tensor Virtual Machine (TVM) component in the NNVM compiler, executing an operator of the intermediate computational graph, and calculating a tensor form of the weight data.
4. The data quantization method according to claim 3 , wherein the step of combining the weight quantization factor and the feature map quantization factor comprises:
combining the weight quantization factor and the feature map quantization factor according to a quantization factor combination calculation formula, wherein the quantization factor combination calculation formula is:
y
w
*
y
f
≈
1
2
n
,
wherein y w represents the weight quantization factor, y f represents the feature map quantization factor, and n represents the quantization parameter.
5. The data quantization method according to claim 4 , wherein the step of separately performing uniform quantization on the weight data and the feature map data of each layer according to the preset linear quantization method and calculating the weight quantization factor and the feature map quantization factor comprises:
calculating an average value of the feature map data of each layer as feature map average data of each layer;
statistically obtaining data distributions of the weight data and the feature map average data of each layer, and calculating corresponding limit values;
limiting the weight data and the feature map average data of each layer within corresponding limit ranges to yield limited data, wherein the limit range is determined according to the corresponding limit value; and
uniformly quantizing the limited data between −127 and +127 of int8 data accuracy, and calculating the weight quantization factor and the feature map quantization factor.
6. The data quantization method according to claim 5 , wherein the step of calculating the corresponding limit values comprises:
calculating a weight limit value of the weight data according to a weight limit value calculation formula, wherein the weight limit value calculation formula is x w =max(|w|), where x w represents the weight limit value, w represents the weight data, and correspondingly, the limit range of the weight data is (−x w , +x w ); and
calculating a feature map limit value of the feature map average data of each layer according to a feature map limit value calculation formula, wherein the feature map limit value calculation formula is x f =max(|F|), where x f represents the feature map limit value, F represents the feature map average data of each layer, and correspondingly, the limit range of the feature map average data of each layer is (−x f , +x f ).
7. The data quantization method according to claim 1 , wherein the quantization parameter is a multiple of 2.
8. A data quantization apparatus, comprising a processor configured to execute a computer program stored in a memory to implement steps of:
performing parsing according to a model file under a current deep learning framework to obtain intermediate computational graph data and weight data that are independent of a hardware environment;
performing, based on the intermediate computational graph data and the weight data, calculation on image data in an input data set through a process indicated by an intermediate computational graph to obtain feature map data;
separately performing uniform quantization on the weight data and the feature map data of each layer according to a preset linear quantization method, and calculating a weight quantization factor and a feature map quantization factor;
combining the weight quantization factor and the feature map quantization factor to obtain a quantization parameter, wherein the quantization parameter is a parameter that makes hardware use shift instead of division; and
writing the quantization parameter and quantized weight data to a bin file according to a hardware requirement so as to generate quantized file data.
9. The data quantization apparatus according to claim 8 , wherein the processor is configured to implement following step:
reordering the quantization parameter and the quantized weight data such that a data format of the quantization parameter and the quantized weight data is a 64-channel parallel format.
10. The data quantization apparatus according to claim 9 , wherein the processor is configured to implement following steps:
parsing the model file by use of a Neural Network Virtual Machine (NNVM) component in an NNVM compiler to obtain the intermediate computational graph data; and
by use of a Tensor Virtual Machine (TVM) component in the NNVM compiler, executing an operator of the intermediate computational graph, and calculating a tensor form of the weight data.
11. The data quantization apparatus according to claim 10 , wherein the processor is configured to implement following steps:
combining the weight quantization factor and the feature map quantization factor according to a quantization factor combination calculation formula, wherein the quantization factor combination calculation formula is:
y
w
*
y
f
≈
1
2
n
,
wherein y w represents the weight quantization factor, y f represents the feature map quantization factor, and n represents the quantization parameter.
12. The data quantization apparatus according to claim 11 , wherein the processor is configured to implement following steps:
calculating an average value of the feature map data of each layer as feature map average data of each layer;
statistically obtaining data distributions of the weight data and the feature map average data of each layer, and calculating corresponding limit values;
limiting the weight data and the feature map average data of each layer within corresponding limit ranges to yield limited data, wherein the limit range is determined according to the corresponding limit value; and
uniformly quantizing the limited data between −127 and +127 of int8 data accuracy, and calculating the weight quantization factor and the feature map quantization factor.
13. The data quantization apparatus according to claim 12 , wherein the processor is configured to implement following steps:
calculating a weight limit value of the weight data according to a weight limit value calculation formula, wherein the weight limit value calculation formula is x w =max(|w|), where x w represents the weight limit value, w represents the weight data, and correspondingly, the limit range of the weight data is (−x w , +x w ); and
calculating a feature map limit value of the feature map average data of each layer according to a feature map limit value calculation formula, wherein the feature map limit value calculation formula is x f =max(|F|), where x f represents the feature map limit value, F represents the feature map average data of each layer, and correspondingly, the limit range of the feature map average data of each layer is (−x f , +x f ).
14. The data quantization apparatus according to claim 8 , wherein the quantization parameter is a multiple of 2.
15. A non-transitory computer readable storage medium configured to store a data quantization program that is executed by a processor to implement steps of:
performing parsing according to a model file under a current deep learning framework to obtain intermediate computational graph data and weight data that are independent of a hardware environment;
performing, based on the intermediate computational graph data and the weight data, calculation on image data in an input data set through a process indicated by an intermediate computational graph to obtain feature map data,
separately performing uniform quantization on the weight data and the feature map data of each layer according to a preset linear quantization method, and calculating a weight quantization factor and a feature map quantization factor;
combining the weight quantization factor and the feature map quantization factor to obtain a quantization parameter, wherein the quantization parameter is a parameter that makes hardware use shift instead of division; and
writing the quantization parameter and quantized weight data to a bin file according to a hardware requirement so as to generate quantized file data.
16. The non-transitory computer readable storage medium according to claim 15 , wherein the data quantization program executed by the processor is configured to implement following step:
reordering the quantization parameter and the quantized weight data such that a data format of the quantization parameter and the quantized weight data is a 64-channel parallel format.
17. The non-transitory computer readable storage medium according to claim 16 , wherein the data quantization program executed by the processor is configured to implement following steps:
parsing the model file by use of a Neural Network Virtual Machine (NNVM) component in an NNVM compiler to obtain the intermediate computational graph data; and
by use of a Tensor Virtual Machine (TVM) component in the NNVM compiler, executing an operator of the intermediate computational graph, and calculating a tensor form of the weight data.
18. The non-transitory computer readable storage medium according to claim 17 , wherein the data quantization program executed by the processor is configured to implement following steps:
combining the weight quantization factor and the feature map quantization factor according to a quantization factor combination calculation formula, wherein the quantization factor combination calculation formula is:
y
w
*
y
f
≈
1
2
n
,
wherein y w represents the weight quantization factor, y f represents the feature map quantization factor, and n represents the quantization parameter.
19. The non-transitory computer readable storage medium according to claim 18 , wherein the data quantization program executed by the processor is configured to implement following steps:
calculating an average value of the feature map data of each layer as feature map average data of each layer;
statistically obtaining data distributions of the weight data and the feature map average data of each layer, and calculating corresponding limit values;
limiting the weight data and the feature map average data of each layer within corresponding limit ranges to yield limited data, wherein the limit range is determined according to the corresponding limit value; and
uniformly quantizing the limited data between −127 and +127 of int8 data accuracy, and calculating the weight quantization factor and the feature map quantization factor.
20. The non-transitory computer readable storage medium according to claim 19 , wherein the data quantization program executed by the processor is configured to implement following steps:
calculating a weight limit value of the weight data according to a weight limit value calculation formula, wherein the weight limit value calculation formula is x w =max(|w|), where x w represents the weight limit value, w represents the weight data, and correspondingly, the limit range of the weight data is (−x w , +x w ); and
calculating a feature map limit value of the feature map average data of each layer according to a feature map limit value calculation formula, wherein the feature map limit value calculation formula is x f =max(|F|), where x f represents the feature map limit value, F represents the feature map average data of each layer, and correspondingly, the limit range of the feature map average data of each layer is (−x f , +x f ).