Iterative reduction of quantization error in neural network model
An electronic device and a controlling method of an electronic device are provided. An electronic device recursively determines a plurality of layers of a neural network model. Weight data of first model information is recursively quantized to obtain a second neural network model. The recursive quantization begins with the weight data and determines an iteration count of a recursion. The recursion operates on error data, quantized weight data, scale data and quantized error data to obtain the iteration count. A first bit-width of the weight data is reduced to a second bit-width of the quantized weight data. The recursion may be performed on a per-layer basis. The weight data may be formulated in a floating-point format and the quantized weight data may be formulated in a fixed point format with an integer number of bits.
1 . An electronic device comprising:
a memory configured to store first model information including information on a plurality of layers of a neural network model and first weight data expressed in a first bit-width; and
a processor configured to:
identify an iteration count of quantization of error data for each layer of the plurality of layers, based on the first model information and an iteration-related loss value,
acquire quantization information including second weight data corresponding to each of the first weight data and expressed in an integer of a second bit-width smaller than the first bit-width by performing recursive quantization according to the iteration count, and
acquire second model information on a reconfigured neural network model by reconfiguring the neural network model based on the first model information, the iteration count, and the second weight data;
acquire temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to an initial iteration count,
acquire a first loss value by performing a feedforward process of the neural network model based on the temporary weight data,
acquire a second loss value for latency of hardware by performing a simulation on the hardware for driving the neural network model based on the temporary weight data, and
identify the iteration count by updating a current iteration count such that a third loss value is decreased, wherein the third loss value is a combination of the first loss value and the second loss value, wherein the third loss value is the iteration-related loss value,
acquire first temporary weight data expressed in the first bit-width by performing first quantization including a quantization and an inverse quantization of the first weight data expressed in the first bit-width,
acquire first error data expressed in the first bit-width based on the first weight data and the first temporary weight data expressed in the first bit-width,
acquire second error data expressed in the second bit-width by performing second quantization including a quantization and an inverse quantization of the first error data expressed in the first bit-width,
determine, as the temporary weight data, second temporary weight data by summing the first temporary weight data, the first error data, and the second error data,
acquire third temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to a first iteration count value acquired by rounding down the current iteration count, and acquire fourth temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to a second iteration count acquired by rounding up the current iteration count, when the current iteration count is not an integer,
interpolate weights corresponding to each other among the third temporary weight data and the fourth temporary weight data, and
determine temporary weight data acquired according to a result of the interpolation as the second temporary weight data.
2 . The electronic device as claimed in claim 1 , wherein the processor is further configured to:
acquire temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to the iteration count,
acquire a loss value by performing a feedforward process of the neural network model based on the temporary weight data, and
acquire the second weight data by updating the first weight data so that the loss value is reduced.
3 . The electronic device as claimed in claim 1 , wherein the quantization information includes first information on the second weight data, first scale information corresponding to the second weight data, second information on second error data corresponding to the second weight data, and second scale information corresponding to the second error data, and
the processor is further configured to acquire the second model information by:
allocating the information on the second weight data and the first scale information to a first operator, and
allocating the information on the second error data and the second scale information to a plurality of second operators.
4 . The electronic device as claimed in claim 1 , wherein the first weight data is implemented through a zero-sum filter having a characteristic that a total sum of first weight values constituting the zero-sum filter is 0, and
when the recursive quantization is performed according to the iteration count, an error value corresponding to each of the first weight values is reflected to a quantization result of each of the first weight values, such that the total sum converges to 0.
5 . The electronic device according to claim 1 , wherein the reconfiguring the neural network model based on the first model information includes adding one or more operators to at least one layer of the plurality of layers of the neural network model.
6 . The electronic device according to claim 4 , wherein the zero-sum filter is a Laplacian filter.
7 . A controlling method of an electronic device, the controlling method comprising:
identifying, based on an iteration-related loss value, an iteration count of quantization for error data for each of a plurality of layers, based on first model information including information on the plurality of layers of a neural network model and first weight data expressed in a first bit-width;
acquiring quantization information including second weight data corresponding to each of the first weight data and expressed in a second bit-width smaller than the first bit-width by performing recursive quantization according to the iteration count; and
acquiring second model information on a reconfigured neural network model by reconfiguring the neural network model based on the first model information, the iteration count, and the second weight data,
wherein the identifying of the iteration count includes:
acquiring temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to an initial iteration count;
acquiring a first loss value by performing a feedforward process of the neural network model based on the temporary weight data;
acquiring a second loss value for latency of hardware by performing a simulation on the hardware for driving the neural network model based on the temporary weight data; and
identifying the iteration count by updating a current iteration count such that a third loss value is decreased, wherein the third loss value is a combination of the first loss value and the second loss value, wherein the third loss value is the iteration-related loss value,
wherein the acquiring of the temporary weight data comprises:
acquiring first temporary weight data expressed in the first bit-width by performing first quantization including a quantization and an inverse quantization of the first weight data expressed in the first bit-width,
acquiring first error data expressed in the first bit-width based on the first weight data and the first temporary weight data expressed in the first bit-width,
acquiring second error data expressed in the second bit-width by performing second quantization including a quantization and an inverse quantization of the first error data expressed in the first bit-width,
determining, as the temporary weight data, second temporary weight data by summing the first temporary weight data, the first error data, and the second error data,
acquiring third temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to a first iteration count acquired by rounding down the current iteration count, and acquiring fourth temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to a second iteration count acquired by rounding up the current iteration count, when the iteration count is not an integer;
interpolating weights corresponding to each other among the third temporary weight data and the fourth temporary weight data; and
determining temporary weight data acquired according to a result of the interpolation as the second temporary weight data.
8 . The controlling method as claimed in claim 7 , wherein the acquiring quantization information further comprises:
acquiring temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to the iteration count,
acquiring a loss value by performing a feedforward process of the neural network model based on the temporary weight data, and
acquiring the second weight data by updating the first weight data so that the loss value is reduced.
9 . The controlling method as claimed in claim 7 , wherein the quantization information includes first information on the second weight data, first scale information corresponding to the second weight data, second information on second error data corresponding to the second weight data, and second scale information corresponding to the error data, and
the acquiring of the second model information further comprises:
allocating the information on the second weight data and the first scale information to a first operator, and
allocating the information on the second error data and the second scale information to a plurality of second operators.
10 . The controlling method as claimed in claim 7 , wherein the first weight data is implemented through a zero-sum filter having a characteristic that a total sum of first weight values constituting the zero-sum filter becomes 0, and
when the recursive quantization is performed according to the iteration count, an error value corresponding to each of the first weight values is reflected to a quantization result of each of the first weight values, such that the total sum converges to 0.
11 . A non-transitory computer readable recording medium including a program for executing a controlling method of an electronic device, wherein the controlling method of the electronic device includes:
identifying an iteration count of quantization for error data for each of a plurality of layers, based on first model information and an iteration-related loss value, including information on the plurality of layers of a neural network model and first weight data expressed in a first bit-width;
acquiring quantization information including second weight data corresponding to each of the first weight data and expressed in a second bit-width smaller than the first bit-width by performing recursive quantization according to the iteration count; and
acquiring second model information on a reconfigured neural network model by reconfiguring the neural network model based on the first model information, the iteration count, and the second weight data,
wherein the identifying of the iteration count includes:
acquiring temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to an initial iteration count;
acquiring a first loss value by performing a feedforward process of the neural network model based on the temporary weight data;
acquiring a second loss value for latency of hardware by performing a simulation on the hardware for driving the neural network model based on the temporary weight data; and
identifying the iteration count by updating a current iteration count such that a third loss value is decreased, wherein the third loss value is a combination of the first loss value and the second loss value, wherein the third loss value is the iteration-related loss value,
wherein the acquiring of the temporary weight data comprises:
acquiring first temporary weight data expressed in the first bit-width by performing first quantization including a quantization and an inverse quantization of the first weight data expressed in the first bit-width,
acquiring first error data expressed in the first bit-width based on the first weight data and the first temporary weight data expressed in the first bit-width,
acquiring second error data expressed in the second bit-width by performing second quantization including a quantization and an inverse quantization of the first error data expressed in the first bit-width,
determining, as the temporary weight data, second temporary weight data by summing the first temporary weight data, the first error data, and the second error data,
acquiring third temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to a first iteration count acquired by rounding down the current iteration count, and acquiring fourth temporary weight data expressed in the first bit-width by performing the recursive quantization on the first weight data according to a second iteration count acquired by rounding up the current iteration count, when the iteration count is not an integer;
interpolating weights corresponding to each other among the third temporary weight data and the fourth temporary weight data; and
determining temporary weight data acquired according to a result of the interpolation as the second temporary weight data.