Quantization method and apparatus for text feature extraction model, and device and storage medium

1 . A quantization method for a text feature extraction model, performed by a processor of a device with a function of quantization for the text feature extraction model, the device comprises a server, a mobile terminal, or a server cluster, and the text feature extraction model is a machine learning model applied to natural language processing, wherein the method comprises:

in a training process of the text feature extraction model, determining, according to a target quantization parameter, a quantization interval corresponding to the target quantization parameter; wherein the quantization interval is an interval obtained by taking a part of floating-point values of the target quantization parameter, the part of the floating-point values of the target quantization parameter is within the quantization interval, and another part of the floating-point values of the target quantization parameter is outside the quantization interval; and wherein the target quantization parameter refers to a target parameter to be quantized in the training process of the text feature extraction model;

constructing a mapping relationship between floating-point values of the target quantization parameter that lie within the quantization interval and fixed-point values of the target quantization parameter, wherein a floating-point value, smaller than a left end point of the quantization interval, of the target quantization parameter is mapped to a quantized minimum fixed-point value, and a floating-point value, larger than a right end point of the quantization interval, of the target quantization parameter is mapped to a quantized maximum fixed-point value; and

performing a quantization operation on the target quantization parameter based on the mapping relationship;

wherein in each time of iterative training of the text feature extraction model, a quantization parameter for constructing the mapping relationship between the floating-point values of the target quantization parameter and the fixed-point values of the target quantization parameter is determined once, and the quantization operation is completed through multiple times of the iterative training of the quantization for the text feature extraction model; wherein the determination of the quantization parameter is performed by the processor of the device, and the processor performs a program stored in a memory of the device to complete the determination; and wherein the mapping relationship between the floating-point values of the target quantization parameter and the fixed-point values of the target quantization parameter is constructed through an algorithm i=q/s+z, where i represents a fixed-point integer, s represents the quantization parameter, z represents an integer value corresponding to a floating-point number q after being quantized, and the floating-point number q is located within the quantization interval;

wherein determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter comprises:

determining an amount 2N of truncated data based on a preset truncation ratio threshold and an amount of the data of the target quantization parameter, wherein N is a product of the preset truncation ratio threshold and the amount of the data of the target quantization parameter; and

determining, based on an (N+1)-th smaller numerical value of the target quantization parameter and an (N+1)-th larger numerical value of the target quantization parameter, the quantization interval corresponding to the target quantization parameter, wherein the quantization interval ranges from the (N+1)-th smaller numerical value to the (N+1)-th larger numerical value, the (N+1)-th smaller numerical value is an (N+1)-th numerical value in a sorting result obtained by sorting the floating-point values of the target quantization parameter in an ascending order, and the (N+1)-th larger numerical value is an (N+1)-th numerical value in a sorting result obtained by sorting the floating-point values of the target quantization parameter in a descending order; and

wherein the quantization interval is [w min , w max ] and the method further comprises: before constructing the mapping relationship, adjusting the quantization interval [w min , w max ] to [−bound*128/127,bound], where bound=max(abs(w cmin ),abs(w cmax )),

wherein in each time of iterative training of the text feature extraction model, the quantization parameter is determined once until an accuracy loss of the text feature extraction model reaches a minimum, or even 0.

2 . The method according to claim 1 , wherein determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter comprises:

determining distribution characteristics of numerical values of the target quantization parameter; and

determining, based on the distribution characteristics of the numerical values, the quantization interval corresponding to the target quantization parameter.

3 . The method according to claim 2 , before determining, based on the distribution characteristics of the numerical values, the quantization interval corresponding to the target quantization parameter, further comprising:

acquiring the preset truncation ratio threshold.

4 . The method according to claim 1 , wherein adjusting the quantization interval comprises:

adjusting the quantization interval to enable a fixed-point value corresponding to a floating-point value 0.0 after being quantized to be 0.

5 . The method according to claim 1 , before determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter, further comprising:

determining the target quantization parameter according to a computation operation of a network layer of the text feature extraction model or a type of a network parameter of the text feature extraction model.

6 . The method according to claim 5 , wherein determining the target quantization parameter according to the computation operation of the network layer of the text feature extraction model comprises:

in a case where the computation operation of the network layer of the text feature extraction model is a product operation, determining an input object used for the product operation as the target quantization parameter; and

in a case where the computation operation of the network layer of the text feature extraction model is a normalization operation, determining an output object of the normalization operation as the target quantization parameter.

7 . The method according to claim 5 , wherein the type of the network parameter comprises a weight matrix and a network bias; and

wherein determining the target quantization parameter according to the type of the network parameter of the text feature extraction model comprises:

in a case where the network parameter is the weight matrix of the text feature extraction model, determining the weight matrix as the target quantization parameter.

8 . An electronic device, wherein the electronic device is a device with a function of quantization for a text feature extraction model, the device comprises a server, a mobile terminal, or a server cluster, and the text feature extraction model is a machine learning model applied to natural language processing, wherein the electronic device comprises:

at least one processing apparatus;

a storage apparatus, configured to store at least one program;

wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus-to implement:

in a training process of the text feature extraction model, determining, according to a target quantization parameter, a quantization interval corresponding to the target quantization parameter; wherein the quantization interval is an interval obtained by taking a part of floating-point values of the target quantization parameter, the part of the floating-point values of the target quantization parameter is within the quantization interval, and another part of the floating-point values of the target quantization parameter is outside the quantization interval; and wherein the target quantization parameter refers to a target parameter to be quantized in the training process of the text feature extraction model;

constructing a mapping relationship between floating-point values of the target quantization parameter that lie within the quantization interval and fixed-point values of the target quantization parameter, wherein a floating-point value, smaller than a left end point of the quantization interval, of the target quantization parameter is mapped to a quantized minimum fixed-point value, and a floating-point value, larger than a right end point of the quantization interval, of the target quantization parameter is mapped to a quantized maximum fixed-point value; and

performing a quantization operation on the target quantization parameter based on the mapping relationship;

wherein in each time of iterative training of the text feature extraction model, a quantization parameter for constructing the mapping relationship between the floating-point values of the target quantization parameter and the fixed-point values of the target quantization parameter is determined once, and the quantization operation is completed through multiple times of the iterative training of the quantization for the text feature extraction model; wherein the determination of the quantization parameter is performed by the at least one processing apparatus of the device, and the at least one processing apparatus performs the at least one program stored in the storage apparatus of the device to complete the determination; and wherein the mapping relationship between the floating-point values of the target quantization parameter and the fixed-point values of the target quantization parameter is constructed through an algorithm i=q/s+z, where i represents a fixed-point integer, s represents the quantization parameter, z represents an integer value corresponding to a floating-point number q after being quantized, and the floating-point number q is located within the quantization interval;

wherein determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter comprises:

determining an amount 2N of truncated data based on a preset truncation ratio threshold and an amount of the data of the target quantization parameter, wherein N is a product of the preset truncation ratio threshold and the amount of the data of the target quantization parameter; and

determining, based on an (N+1)-th smaller numerical value of the target quantization parameter and an (N+1)-th larger numerical value of the target quantization parameter, the quantization interval corresponding to the target quantization parameter, wherein the quantization interval ranges from the (N+1)-th smaller numerical value to the (N+1)-th larger numerical value, the (N+1)-th smaller numerical value is an (N+1)-th numerical value in a sorting result obtained by sorting the floating-point values of the target quantization parameter in an ascending order, and the (N+1)-th larger numerical value is an (N+1)-th numerical value in a sorting result obtained by sorting the floating-point values of the target quantization parameter in a descending order; and

wherein the quantization interval is [W cmin , W cmax ], and the at least one processing apparatus is further configured to: adjust the quantization interval [w cmin ,w cmax ] to [−bound*128/127,bound], where bound=max(abs(w cmin ),abs(w cmax )),

wherein in each time of iterative training of the text feature extraction model, the quantization parameter is determined once until an accuracy loss of the text feature extraction model reaches a minimum, or even 0.

9 . The electronic device of claim 8 , wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to implement determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter in the following way:

determining distribution characteristics of numerical values of the target quantization parameter; and

determining, based on the distribution characteristics of the numerical values, the quantization interval corresponding to the target quantization parameter.

10 . The electronic device of claim 9 , wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to, before implementing determining, based on the distribution characteristics of the numerical values, the quantization interval corresponding to the target quantization parameter, further implement:

acquiring the preset truncation ratio threshold.

11 . The electronic device of claim 8 , wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to, implement adjusting the quantization interval in the following way:

adjusting the quantization interval to enable a fixed-point value corresponding to a floating-point value 0.0 after being quantized to be 0.

12 . The electronic device of claim 8 , wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to, before implementing determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter, further implement:

determining the target quantization parameter according to a computation operation of a network layer of the text feature extraction model or a type of a network parameter of the text feature extraction model.

13 . The electronic device of claim 12 , wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to implement determining the target quantization parameter according to the computation operation of the network layer of the text feature extraction model in the following way:

in a case where the computation operation of the network layer of the text feature extraction model is a product operation, determining an input object used for the product operation as the target quantization parameter; and

in a case where the computation operation of the network layer of the text feature extraction model is a normalization operation, determining an output object of the normalization operation as the target quantization parameter.

14 . The electronic device of claim 12 , wherein the type of the network parameter comprises a weight matrix and a network bias; and

wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to implement determining the target quantization parameter according to the type of the network parameter of the text feature extraction model in the following way:

in a case where the network parameter is the weight matrix of the text feature extraction model, determining the weight matrix as the target quantization parameter.

15 . A non-transitory computer-readable medium storing a computer program, wherein the computer program, when executed by a processor, implement a quantization method for a text feature extraction model, the quantization method being performed by the processor of a device with a function of quantization for the text feature extraction model, the device comprises a server, a mobile terminal, or a server cluster, and the text feature extraction model is a machine learning model applied to natural language processing, wherein the quantization method comprises:

in a training process of the text feature extraction model, determining, according to a target quantization parameter, a quantization interval corresponding to the target quantization parameter; wherein the quantization interval is an interval obtained by taking a part of floating-point values of the target quantization parameter, the part of the floating-point values of the target quantization parameter is within the quantization interval, and another part of the floating-point values of the target quantization parameter is outside the quantization interval; and wherein the target quantization parameter refers to a target parameter to be quantized in the training process of the text feature extraction model;

constructing a mapping relationship between floating-point values of the target quantization parameter that lie within the quantization interval and fixed-point values of the target quantization parameter, wherein a floating-point value, smaller than a left end point of the quantization interval, of the target quantization parameter is mapped to a quantized minimum fixed-point value, and a floating-point value, larger than a right end point of the quantization interval, of the target quantization parameter is mapped to a quantized maximum fixed-point value; and

performing a quantization operation on the target quantization parameter based on the mapping relationship;

wherein in each time of iterative training of the text feature extraction model, a quantization parameter for constructing the mapping relationship between the floating-point values of the target quantization parameter and the fixed-point values of the target quantization parameter is determined once, and the quantization operation is completed through multiple times of the iterative training of the quantization for the text feature extraction model; wherein the determination of the quantization parameter is performed by the processor of the device, and the processor performs the computer program stored in the non-transitory computer-readable medium to complete the determination;

and wherein the mapping relationship between the floating-point values of the target quantization parameter and the fixed-point values of the target quantization parameter is constructed through an algorithm i=q/s+z, where i represents a fixed-point integer, s represents the quantization parameter, z represents an integer value corresponding to a floating-point number q after being quantized, and the floating-point number q is located within the quantization interval;

wherein determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter comprises:

determining an amount 2 N of truncated data based on a preset truncation ratio threshold and an amount of the data of the target quantization parameter, wherein N is a product of the preset truncation ratio threshold and the amount of the data of the target quantization parameter; and

determining, based on an (N+1)-th smaller numerical value of the target quantization parameter and an (N+1)-th larger numerical value of the target quantization parameter, the quantization interval corresponding to the target quantization parameter, wherein the quantization interval ranges from the (N+1)-th smaller numerical value to the (N+1)-th larger numerical value, the (N+1)-th smaller numerical value is an (N+1)-th numerical value in a sorting result obtained by sorting the floating-point values of the target quantization parameter in an ascending order, and the (N+1)-th larger numerical value is an (N+1)-th numerical value in a sorting result obtained by sorting the floating- point values of the target quantization parameter in a descending order; and

wherein the quantization interval is [w cmin , w cmax ], and the method further comprises: before constructing the mapping relationship, adjusting the quantization interval [w cmin , w cmax ]to [−bound*128/127,bound], where bound=max(abs(W cmin ),abs(w cmax )),

wherein in each time of iterative training of the text feature extraction model, the quantization parameter is determined once until an accuracy loss of the text feature extraction model reaches a minimum, or even 0.

16 . The non-transitory computer-readable medium of claim 15 , wherein the computer program, when executed by the processing apparatus, implement determining, according to the target quantization parameter, the quantization interval corresponding to the target quantization parameter in the following way:

determining distribution characteristics of numerical values of the target quantization parameter; and

determining, based on the distribution characteristics of the numerical values, the quantization interval corresponding to the target quantization parameter.