Quantizing machine learning models with balanced resolution via damped encoding

1. A method for performing an operation of quantizing weights and values of a machine learning model during an inference phase of the machine learning model, the method comprising:

acquiring a set of floating-point values of the weights and values of the machine learning model before the set of floating-point values being processed by a processor for performing the operation;

acquiring a normalization factor determined using the set of floating-point values and a damped value of a damped value sequence, wherein the damped value sequence comprises a plurality of damped values, and spacings between adjacent damped values in the damped value sequence are non-linear;

assigning a quantized value for each floating-point value of the set of floating-point values based on the damped value sequence and the normalization factor, wherein the damped value sequence is computed based on a damped Fibonacci recurrence relationship and a damping weight; and

sending the quantized value to the processor for performing the operation of quantizing the weights and values of the machine learning model.

2. The method of claim 1 , further comprising determining the damped value from the damped value sequence using the damped value sequence and a number of quantization bits.

3. The method of claim 2 , wherein determining the damped value from the damped value sequence comprises using the largest damped value from the damped value sequence based on the number of quantization bits.

4. The method of claim 1 , wherein assigning the quantized value for the each floating-point value of the set of floating-point values comprises:

assigning the quantized value for the each floating-point value in the set of floating-point values based on a comparison of adjacent damped values of the damped value sequence using the normalization factor.

5. The method of claim 1 , wherein the damped value sequence is computed based on

DF

i

⁡

(

Original

)

=

DF

i

-

1

+

⌈

1

N

*

DF

i

-

2

⌉

,

wherein N is a damping weight.

6. The method of claim 1 , wherein the normalization factor is determined using a maximum value from the set of floating-point values, a minimum value from the set of floating-point values, and the damped value.

7. The method of claim 6 , wherein the normalization factor is determined based on:

α

=

(

max

-

min

)

DF

⁡

(

2

k

-

1

)

,

wherein α is the normalization factor, max is the maximum value, min is the minimum value, k is a number of quantization bits, and DF (2 k −1) is the damped value.

8. The method of claim 6 , wherein assigning the quantized value for the each floating-point value of the set of floating-point values is based on one of

x

_

=

{

∃

i

⁢

⁢

such

⁢

⁢

that

⁢

⁢

α

*

DF

⁡

(

2

i

-

1

)

≤

(

x

-

min

)

<

α

*

DF

⁡

(

2

i

)

,

return

⁢

⁢

(

2

i

-

1

)

}

,

⁢

⁢

or

x

_

=

{

∃

i

⁢

⁢

such

⁢

⁢

that

⁢

⁢

DF

⁡

(

2

i

-

1

)

≤

x

-

min

α

<

DF

⁡

(

2

i

)

,

return

⁢

⁢

(

2

i

-

1

)

}

,

wherein x is the quantized value, α is the normalization factor, x is a floating-point value from the set of floating-point values, min is the minimum value, and DF(2 i −1) and DF(2 i ) are adjacent damped values of the damped value sequence.

9. A non-transitory computer-readable medium that stores a set of computer executable instructions that are executable by a processor of a device to cause the device to perform a method for performing an operation of quantizing weights and values of a machine learning model during an inference phase of the machine learning model, the method comprising:

acquiring a set of floating-point values of the weights and values of the machine learning model before the set of floating-point values being processed by a processor for performing an operation in the inference phase;

acquiring a normalization factor determined using the set of floating-point values and a damped value of a damped value sequence, wherein the damped value sequence comprises a plurality of damped values, and spacings between adjacent damped values in the damped value sequence are non-linear;

assigning a quantized value for each floating-point value of the set of floating-point values based on the damped value sequence and the normalization factor, wherein the damped value sequence is computed based on a damped Fibonacci recurrence relationship and a damping weight; and

sending the quantized value to the processor for performing the operation of quantizing the weights and values of the machine learning model.

10. The computer-readable medium of claim 9 , wherein the set of computer executable instructions that are executable by the processor of the device to cause the device to further perform:

determining the damped value from the damped value sequence using the damped value sequence and a number of quantization bits.

11. The computer-readable medium according to claim 10 , wherein determining the damped value from the damped value sequence comprises using the largest damped value from the damped value sequence based on the number of quantization bits.

12. The computer-readable medium according to claim 9 , wherein assigning the quantized value for the each floating-point value of the set of floating-point values comprises:

assigning the quantized value for the each floating-point value in the set of floating-point values based on a comparison of adjacent damped values of the damped value sequence using the normalization factor.

13. The computer readable medium according to claim 9 , wherein the damped value sequence is computed based on

DF

i

⁡

(

Original

)

=

DF

i

-

1

+

⌈

1

N

*

DF

i

-

2

⌉

,

wherein N is a damping weight.

14. The computer-readable medium according to claim 9 , wherein the normalization factor is determined using a maximum value from the set of floating-point values, a minimum value from the set of floating-point values, and the damped value.

15. The computer-readable medium according to claim 14 , wherein the normalization factor is determined based on:

α

=

(

max

-

min

)

DF

⁡

(

2

k

-

1

)

,

wherein α is the normalization factor, max is the maximum value, min is the minimum value, k is a number of quantization bits, and DF (2 k −1) is the damped value.

16. The computer readable medium according to claim 14 , wherein assigning the quantized value for the each floating-point value of the set of floating-point values is based on one of

x

_

=

{

∃

i

⁢

⁢

such

⁢

⁢

that

⁢

⁢

α

*

DF

⁡

(

2

i

-

1

)

≤

(

x

-

min

)

<

α

*

DF

⁡

(

2

i

)

,

return

⁢

⁢

(

2

i

-

1

)

}

,

⁢

⁢

or

x

_

=

{

∃

i

⁢

⁢

such

⁢

⁢

that

⁢

⁢

DF

⁡

(

2

i

-

1

)

≤

x

-

min

α

<

DF

⁡

(

2

i

)

,

return

⁢

⁢

(

2

i

-

1

)

}

,

wherein x is the quantized value, a is the normalization factor, x is a floating-point value from the set of floating-point values, min is the minimum value, and DF(2 i −1) and DF(2 i ) are adjacent damped values of the damped value sequence.

17. A quantizer for performing an operation of quantizing weights and values of a machine learning model during an inference phase of the machine learning model, the quantizer comprising:

one or more memories configured to store one or more floating-point values of a set of floating-point values of the weights and values of the machine learning model, and a normalization factor determined using the set of floating-point values and a damped value of a damped value sequence, wherein the damped value sequence comprises a plurality of damped values, and spacings between adjacent damped values in the damped value sequence are non-linear; and

quantization circuitry configured to:

acquire the normalization factor and the one or more floating-point values from the one or more memories before the one or more floating-point values being processed by an apparatus for performing the operation in the inference phase;

assign a quantized value for each floating-point value of the set of floating-point values based on the damped value sequence and the normalization factor, wherein the damped value sequence is computed based on a damped Fibonacci recurrence relationship and a damping weight; and

send the quantized value to the apparatus for performing the operation of quantizing the weights and values of the machine learning model.

18. The quantizer of claim 17 , wherein the damped value from the damped value sequence is determined using the damped value sequence and a number of quantization bits.