Method and apparatus with optimization for deep learning model
A method with quantization for a deep learning model includes: determining a second model by quantizing a first model based on a quantization parameter; determining a real value of multi optimization target parameter by testing the second model; calculating a loss function based on the real value of the multi optimization target parameter, an expected value of the multi optimization target parameter, and a constraint value of the multi optimization target parameter; updating the quantization parameter based on the loss function and using the second model as the first model; iteratively executing the foregoing operations until a preset condition is satisfied; and in response to the preset condition being satisfied, determining an optimal quantization parameter and using, as a final quantization model, the first model that executes quantization based on the optimal quantization parameter.
1 . A method with quantization for a deep learning model, comprising:
determining a second model by quantizing a first model based on a quantization parameter;
determining a real value of a multi optimization target parameter by testing the second model;
determining a loss function based on the real value of the multi optimization target parameter, an expected value of the multi optimization target parameter, and by assigning a penalty in response to a real value of a specific target parameter of the multi optimization target parameter exceeding a constraint value of the multi optimization target parameter, wherein a value of the penalty is determined based on a penalty factor and a difference value between the real value of the specific target parameter and the constraint value;
updating the quantization parameter based on the loss function and determining the second model as the first model;
iteratively executing the foregoing operations until a preset condition is satisfied; and
in response to the preset condition being satisfied, determining an optimal quantization parameter and using, as a final quantization model, the first model that is quantized based on the optimal quantization parameter,
wherein the determining of the second model comprises:
determining a simulation quantization model using the first model, for mapping the quantization parameter with each operator to be quantized in the first model; and
determining the second model by executing model reconfiguration on the simulation quantization model based on a quantization coefficient, the quantization coefficient comprising a scale factor and a zero point.
2 . The method of claim 1 , wherein the determining of the simulation quantization model comprises:
determining the simulation quantization model by inserting a simulation quantization operator into the each operator to be quantized in the first model, wherein the each operator to be quantized comprises a weight or an activation; and
executing quantization configuration on the simulation quantization model based on the quantization parameter, wherein the quantization parameter is mapped with the simulation quantization operator of each layer in the simulation quantization model, and
the method further comprises:
determining the quantization coefficient based on the simulation quantization model determined in response to the quantization configuration.
3 . The method of claim 1 , wherein the multi optimization target parameter comprises any one or any combination of any two or more parameters among accuracy, a size of the quantization model, energy consumption, and inference latency.
4 . The method of claim 1 , wherein the determining of the loss function comprises:
determining the loss function based on a difference value between the real value and the expected value of the multi optimization target parameter and a difference value between the real value and the constraint value of the multi optimization target parameter.
5 . The method of claim 4 , wherein the loss function is represented as
loss
t
+
c
=
∑
j
=
1
M
F
(
o
j
,
c
j
,
t
j
,
w
j
,
λ
j
)
=
∑
j
=
1
M
w
j
×
Δ
t
j
2
+
∑
j
=
1
M
λ
j
×
(
max
(
0
,
Δ
c
j
)
)
2
,
wherein t denotes the expected value, and t∈R + denotes an expected value of a single optimization target parameter,
c denotes the constraint value, and c∈R + denotes a constraint value of the single optimization target parameter,
o denotes the real value, and o∈R + denotes a real value of a specific optimization target parameter of a current quantization model,
Δ
tj
=
t
j
-
o
j
denotes the difference value between the real value and the expected value,
Δ
cj
=
c
j
-
o
j
denotes the difference value between the real value and the constraint value,
wj denotes a weighting factor, w∈R + , and
Δ
tj
2
is an optimization term, wherein, for minimizing a loss, an importance of each optimization target parameter is adjusted by the weighting factor,
a term
w
j
×
Δ
tj
2
is used to evaluate each optimization target parameter by a final result,
λ j denotes the penalty factor and λ∈R + ,
(max (0, Δ cj )) 2 is a penalty term, wherein, in response to the real value of the specific optimization target parameter of the second model exceeding the constraint value, the penalty is assigned such that each optimization target parameter reaches a constraint, and
M denotes a total number of optimization target parameters.
6 . The method of claim 1 , wherein the updating of the quantization parameter comprises:
determining and recording a new quantization parameter set of the second model based on a function value of the loss function and a target algorithm, wherein the target algorithm comprises a Bayesian optimization algorithm; and
replacing a current quantization parameter of the second model using the new quantization parameter set.
7 . The method of claim 6 , wherein the determining of the optimal quantization parameter comprises:
in response to the preset condition being satisfied, using, as the optimal quantization parameter, a set that minimizes the function value of the loss function among a plurality of quantization parameter sets recorded by screening,
wherein the preset condition is satisfied in response to the number of iterations of the operations reaching a preset number, or in response to an iteration time reaching a preset iteration time.
8 . The method of claim 1 , wherein a precision type corresponding to the quantization parameter comprises any one or any combination of any two or more categories among INT4, INT8, and INT16.
9 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 .
10 . An electronic device, comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, configure the one or more processors to perform the method of claim 1 .
11 . An apparatus with quantization for a deep learning model, comprising:
one or more processors configured to:
determine a second model by quantizing a first model based on a quantization parameter;
determine a real value of a multi optimization target parameter by testing the second model;
determine a loss function based on the real value of the multi optimization target parameter, an expected value of the multi optimization target parameter, and by assigning a penalty in response to a real value of a specific target parameter of the multi optimization target parameter exceeding a constraint value of the multi optimization target parameter, wherein a value of the penalty is determined based on a penalty factor and a difference value between the real value of the specific target parameter and the constraint value;
update the quantization parameter based on the loss function and use the second model as the first model; and
in response to a result of updating the quantization parameter satisfying a preset condition, determine an optimal quantization parameter and use, as a final quantization model, the first model that is quantized based on the optimal quantization parameter, wherein the determining of the second model comprises:
determining a simulation quantization model using the first model, for mapping the quantization parameter with each operator to be quantized in the first model; and
determining the second model by executing model reconfiguration on the simulation quantization model based on a quantization coefficient, the quantization coefficient comprising a scale factor and a zero point.
12 . The apparatus of claim 11 , wherein, for determining of the simulation quantization, the one or more processors are configured to:
determine the simulation quantization model by inserting a simulation quantization operator into the each operator to be quantized in the first model, wherein the each operator to be quantized comprises a weight or an activation; and
execute quantization configuration on the simulation quantization model based on the quantization parameter, wherein the quantization parameter is mapped with the simulation quantization operator of each layer in the simulation quantization model, and;
the one or more processors are further configured to:
determine the quantization coefficient based on the simulation quantization model determined in response to the quantization configuration.
13 . The apparatus of claim 11 , wherein the multi optimization target parameter comprises any one or any combination of any two or more parameters among accuracy, a size of the quantization model, energy consumption, and inference latency.
14 . The apparatus of claim 11 , wherein, for the determining of the loss function, the one or more processors are configured to:
determine the loss function based on a difference value between the real value and the expected value of the multi optimization target parameter and a difference value between the real value and the constraint value of the multi optimization target parameter.
15 . The apparatus of claim 11 , wherein, for the updating of the quantization parameter, the one or more processors are configured to:
determine and record a new quantization parameter set of the second model based on a function value of the loss function and a target algorithm, wherein the target algorithm comprises a Bayesian optimization algorithm; and
replace a current quantization parameter of the second model using the new quantization parameter set.
16 . The apparatus of claim 11 , wherein a precision type corresponding to the quantization parameter comprises any one or any combination of any two or more categories among INT4, INT8, and INT16.
17 . An apparatus with quantization for a deep learning model, comprising:
one or more processors configured to:
determine a second model by quantizing, for each of a plurality of layers of a first model, either one or both of an activation value and a weight based on a quantization parameter;
determine a value of a multi optimization target parameter based on the second model;
determine a loss based on the value of the multi optimization target parameter, an expected value of the multi optimization target parameter, and by assigning a penalty in response to a real value of a specific target parameter of the multi optimization target parameter exceeding a constraint value of the multi optimization target parameter, wherein a value of the penalty is determined based on a penalty factor and a difference value between the real value of the specific target parameter and the constraint value;
update the quantization parameter based on the loss and use the second model as the first model; and
update the first model to execute quantization based on the updated quantization parameter,
wherein the determining of the second model comprises:
determining a simulation quantization model using the first model, for mapping the quantization parameter with each operator to be quantized in the first model; and
determining the second model by executing model reconfiguration on the simulation quantization model based on a quantization coefficient, the quantization coefficient comprising a scale factor and a zero point.
18 . The apparatus of claim 17 , wherein the one or more processors are configured to determine the value of the multi optimization target parameter by inputting a test data set to the second model.
19 . The apparatus of claim 17 , wherein, for each of the layers of the first model, the quantization parameter corresponds to a precision type of either one or both of the activation value the weight.
20 . The apparatus of claim 17 , wherein, for the determining of the second model, the one or more processors are configured to convert, for each of the layers of the first model, a floating-point operator into an integer operator having a precision of the quantization parameter.