Neural network based training method, inference method and apparatus
Disclosed are a neural network-based training method, inference method and apparatus. The neural network-based inference method includes receiving a quantization level for quantizing a weight of a neural network and an activation value that is processed by the neural network, receiving a weight quantized based on the quantization level, generating a quantized activation value by quantizing the activation value based on the quantization level, and performing inference based on the quantized weight and the quantized activation value.
1 . A neural network-based inference method, comprising:
receiving a quantization level for quantizing a weight of a neural network and for quantizing an activation value that is processed by the neural network;
receiving a weight quantized based on the quantization level;
generating a quantized activation value by quantizing the activation value based on the quantization level; and
performing inference based on the quantized weight and the quantized activation value, including splitting the quantized activation value based on the quantization level to obtain split activation values and performing the inference based on the quantized weight and the split activation values.
2 . The neural network-based inference method of claim 1 , wherein the obtaining of the split activation values comprises obtaining the split activation values by splitting the quantized activation value based on a ratio of a first quantization level included in the quantization level and a second quantization level included in the quantization level.
3 . The neural network-based inference method of claim 2 , wherein the obtaining of the split activation values by splitting the quantized activation value based on the ratio of the first quantization level and the second quantization level comprises splitting the quantized activation value into a number of activation values corresponding to the ratio.
4 . The neural network-based inference method of claim 2 , wherein the ratio is an integer.
5 . The neural network-based inference method of claim 1 , wherein
the quantized weight comprises weights split based on the quantization level, and
the performing comprises:
obtaining channels by performing an operation based on the split weights and the quantized activation value; and
performing the inference by merging the channels.
6 . A neural network-based inference apparatus, comprising:
a receiver processor configured to receive a quantization level for quantizing a weight of a neural network and for quantizing an activation value that is processed by the neural network, and to receive a weight quantized based on the quantization level; and
a processor configured to generate a quantized activation value by quantizing the activation value based on the quantization level, and to perform inference based on the quantized weight and the quantized activation value, including
splitting the quantized activation value based on the quantization level to obtain split activation values and performing the inference based on the quantized weight and the split activation values.
7 . The neural network-based inference apparatus of claim 6 , wherein the processor is further configured to obtain the split activation values by splitting the quantized activation value based on a ratio of a first quantization level included in the quantization level and a second quantization level included in the quantization level.
8 . The neural network-based inference apparatus of claim 7 , wherein the processor is further configured to obtain the split the quantized activation value into a number of activation values corresponding to the ratio.
9 . The neural network-based inference apparatus of claim 7 , wherein the ratio is an integer.
10 . The neural network-based inference apparatus of claim 6 , wherein
the quantized weight comprises weights split based on the quantization level, and
the processor is further configured to obtain channels by performing an operation based on the split weights and the quantized activation value, and to perform the inference by merging the channels.
11 . A neural network-based training method, comprising:
quantizing a weight of a neural network and quantizing an activation value that is processed by the neural network based on a first quantization level and a second quantization level;
generating a first channel by performing an operation included in the neural network based on a first quantized weight and a first quantized activation value;
generating a second channel by performing an operation included in the neural network based on a second quantized weight and the first quantized activation value; and
selecting one of the first channel and the second channel based on a channel selection mask.
12 . The neural network-based training method of claim 11 , further comprising:
determining the first quantization level and the second quantization level based on a loss of the neural network.
13 . The neural network-based training method of claim 12 , wherein the determining comprises:
determining the first quantization level based on the loss; and
determining the second quantization level based on the first quantization level.
14 . The neural network-based training method of claim 13 , wherein the determining of the second quantization level based on the first quantization level comprises determining the first quantization level to be an integer multiple of the second quantization level.
15 . The neural network-based training method of claim 11 , wherein the selecting comprises:
generating the channel selection mask; and
selecting one of the first channel and the second channel based on the channel selection mask.
16 . The neural network-based training method of claim 15 , wherein the generating of the channel selection mask comprises generating the channel selection mask based on a deviation between a first output of the neural network based on the first quantization level and a second output of the neural network based on the second quantization level.
17 . The neural network-based training method of claim 15 , wherein the generating of the channel selection mask comprises:
generating the channel selection mask based on a sensitivity indicating a loss of the neural network for a deviation between a weight based on the first quantization level and a weight based on the second quantization level; or
generating the channel selection mask based on a sensitivity indicating a loss of the neural network for a deviation between an activation value based on the first quantization level and an activation value based on the second quantization level.
18 . The neural network-based training method of claim 12 , wherein the generating of the channel selection mask comprises training the channel selection mask based on the loss of the neural network.