Granted Patent B2

US 12,566,940 · App. 17/192,048 · Granted Mar 3, 2026

Inventors: Gunhee Kim (Seoul, KR); Seungwon Lee (Hwaseong-si, KR)

Assignee: Samsung Electronics Co., Ltd.

G06N3/04

Abstract

Claims (67)

1 . A computer-implemented method of quantizing parameters of a neural network, the method comprising:

acquiring, from a first memory, a parameter of a floating-point format used in a process of inferring by the neural network;

quantizing, by a quantization device, based on statistics of a weight included in the parameter, the weight into a fixed-point format;

determining, based on statistics of an activation of one or more layers configuring the neural network included in the parameter, a dynamic range of the activation, wherein the determining of the dynamic range of the activation comprises:

calculating a relational expression between an Lp norm of an output activation and an Lp norm of an input activation of the one or more layers based on the parameter; and

inferring an Lp norm of an output activation of the one or more layers based on the relational expression and an Lp norm of an output activation of a previous layer of the one or more layers;

quantizing, by the quantization device, based on statistics of input data of the neural network, the input data into a fixed-point format;

inputting, to the neural network, the quantized input data stored in a second memory;

performing, by a NPU, an operation between a quantized input activation of a current layer of the neural network and a quantized weight of the current layer and generating an output activation of the current laver; and

quantizing the output activation of the current layer based on the dynamic range of the activation, into the fixed-point format.

2 . The method of claim 1 , wherein the quantizing of the weight into the fixed-point format comprises:

calculating an Lp norm of the weight;

determining a dynamic range of the weight based on the Lp norm of the weight; and

quantizing the weight into the fixed-point format based on the dynamic range of the weight.

3 . The method of claim 1 , wherein the determining of the dynamic range of the activation comprises: determining a dynamic range of the output activation of the one or more layers based on the Lp norm of the output activation of the one or more layers.

4 . The method of claim 1 , wherein the determining of the dynamic range of the activation comprises:

classifying the one or more layers into a first layer comprising a normalization layer and a second layer that does not comprise the normalization layer;

inferring an Lp norm of an output activation of the first layer based on a normalization parameter of the first layer included in the parameter; and

inferring an Lp norm of an output activation of the second layer based on the parameter and the Lp norm of the output activation of the first layer.

5 . The method of claim 4 , wherein the inferring of the Lp norm of the output activation of the second layer comprises:

calculating a relational expression between an Lp norm of an input activation and an Lp norm of the output activation of the second layer based on the parameter;

determining the Lp norm of the input activation of the second layer based on the Lp norm of the output activation of the first layer; and

inferring the Lp norm of the output activation of the second layer based on the Lp norm of the input activation of the second layer and the relational expression.

6 . The method of claim 4 , wherein the determining of the dynamic range of the activation comprises:

determining a dynamic range of the output activation of the first layer based on the Lp norm of the output activation of the first layer; and

determining a dynamic range of the output activation of the second layer based on the Lp norm of the output activation of the second layer.

7 . A neural network quantization apparatus for quantizing parameters of a neural network, the apparatus comprising:

a first memory:

a second memory; and

a processor configured to:

acquire, from the first memory, a parameter of a floating-point format used in a process of inferring by the neural network,

quantize, based on statistics of a weight included in the parameter, the weight into a fixed-point format,

determine, based on statistics of an activation of one or more layers configuring the neural network included in the parameter, a dynamic range of the activation, wherein the processor is configured to:

calculate a relational expression between an Lp norm of an output activation and an Lp norm of an input activation of the one or more layers based on the parameter, and

infer an Lp norm of an output activation of the one or more layers based on the relational expression and an Lp norm of an output activation of a previous layer of the one or more layers,

quantize, based on statistics of input data of the neural network, the input data into a fixed-point format;

input, to the neural network, the quantized input data stored in the second memory;

perform an operation between a quantized input activation of a current layer of the neural network and a quantized weight of the current layer and generating an output activation of the current layer; and

quantize the output activation of the current layer, based on the dynamic range of the activation, into the fixed-point format.

8 . The neural network quantization apparatus of claim 7 , wherein the processor is configured to:

calculate an Lp norm of the weight,

determine a dynamic range of the weight based on the Lp norm of the weight, and

quantize the weight into the fixed-point format based on the dynamic range of the weight.

9 . The neural network quantization apparatus of claim 7 , wherein the processor is configured to determine a dynamic range of the output activation of the one or more layers based on the Lp norm of the output activation of the one or more layers.

10 . The neural network quantization apparatus of claim 7 , wherein the processor is configured to:

classify the one or more layers into a first layer comprising a normalization layer and a second layer that does not comprise the normalization layer,

infer an Lp norm of an output activation of the first layer based on a normalization parameter of the first layer included in the parameter, and

infer an Lp norm of an output activation of the second layer based on the parameter and the Lp norm of the output activation of the first layer.

11 . The neural network quantization apparatus of claim 10 , wherein the processor is configured to:

calculate a relational expression between an Lp norm of an input activation and an Lp norm of the output activation of the second layer based on the parameter,

determine the Lp norm of the input activation of the second layer based on the Lp norm of the output activation of the first layer, and

infer an Lp norm of the output activation of the second layer based on the Lp norm of the input activation of the second layer and the relational expression.

12 . The neural network quantization apparatus of claim 10 , wherein the processor is configured to:

determine a dynamic range of the output activation of the first layer based on the Lp norm of the output activation of the first layer, and

determine a dynamic range of the output activation of the second layer based on the Lp norm of the output activation of the second layer.

13 . A neural processing unit (NPU) comprising:

a memory comprising an Lp norm of a weight, an Lp norm of an output activation of layers, and an Lp norm of input data; and

a neural network quantization apparatus configured to:

determine a dynamic range of the weight based on the Lp norm of the weight,

quantize the weight into a fixed-point format based on the dynamic range of the weight:

determine a dynamic range of the output activation of the layers based on the Lp norm of the output activation of the layers, wherein the determining of the dynamic range of the activation comprises:

calculating a relational expression between an Lp norm of an output activation and an Lp norm of an input activation of the one or more layers based on the parameter; and

inferring an Lp norm of an output activation of the one or more layers based on the relational expression and an Lp norm of an output activation of a previous layer of the one or more lavers:

determine a dynamic range of the input data based on the Lp norm of the input data:

quantize the input data into a fixed-point format based on the dynamic range of the input data; and

an arithmetic logic unit (ALU) configured to perform an operation between a quantized input activation of a current layer of the neural network and a quantized weight of the current layer and generate an output activation of the current layer,

wherein the neural network quantization apparatus is configured to quantize the output activation of the current layer based on the dynamic range of the output activation.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (51)