Method and device with neural network model
View Patent ↗A device includes: a comparator; an exclusive-NOR (XNOR) gate; an accumulator; and a multiplication and accumulation (MAC) operator, wherein a basic block of a neural network comprises a first batch normalization layer, a quantization layer, a convolution layer, an active layer, and a second batch normalization layer, and wherein the basic block is driven by the device by a combination of a first batch normalization operation, a sign function operation, a bitwise convolution operation, an activation function operation, a second batch normalization operation, and a residual connection operation.
1. A device, the device comprising:
a comparator;
an exclusive-NOR (XNOR) gate;
an accumulator; and
a multiplication and accumulation (MAC) operator,
wherein a basic block of a neural network comprises a first batch normalization layer, a quantization layer, a convolution layer, an active layer, and a second batch normalization layer, and
wherein the basic block is driven by the device by a combination of a first batch normalization operation, a sign function operation, a bitwise convolution operation, an activation function operation, a second batch normalization operation, and a residual connection operation.
2. The device of claim 1 , wherein the comparator is configured to quantize input data by comparing a result of performing the sign function operation on transformed input data with a predetermined value.
3. The device of claim 2 , wherein
the second batch normalization operation comprises a first portion and a second portion, and
the transformed input data reflects a second portion of first batch normalized input data determined using the first batch normalization operation.
4. The device of claim 1 , wherein the XNOR gate and the accumulator are configured to perform the bitwise convolution operation.
5. The device of claim 1 , wherein
the second batch normalization operation comprises a first portion and a second portion, and
the MAC operator is configured to perform the activation function operation, the residual connection operation, and an operation for the first portion.
6. The device of claim 5 , wherein the MAC operator is configured to:
determine output data by applying the activation function to a bitwise convolution operation result of the bitwise convolution operation;
multiply the first portion by the output data; and
perform an operation of adding input data.
7. The device of claim 6 , wherein the activation function comprises:
a rectified linear unit (ReLU) function.
8. A processor-implemented method, the method comprising:
quantizing input data by comparing a result of performing a sign function operation on transformed input data with a predetermined value using a comparator;
performing a bitwise convolution operation using an exclusive-NOR (XNOR) gate and an accumulator; and
performing an activation function operation and a residual connection operation using a multiplication and accumulation (MAC) operator.
9. The method of claim 8 , wherein
a second batch normalization operation comprises a first portion and a second portion, and
the transformed input data reflects a second portion of first batch normalized input data determined using the first batch normalization operation.
10. The method of claim 8 , wherein
the second batch normalization operation comprises a first portion and a second portion, and
performing of operations using the MAC operator comprises performing the activation function operation, the residual connection operation, and an operation for the first portion.
11. The method of claim 10 , wherein the performing of the operations using the MAC operator comprises:
determining output data by applying the activation function to a bitwise convolution operation result of the bitwise convolution operation;
multiplying the first portion by the output data; and
performing an operation of adding input data.
12. The method of claim 11 , wherein the activation function comprises:
a rectified linear unit (ReLU) function.
13. The method of claim 8 , wherein
a basic block of a neural network comprises a first batch normalization layer, a quantization layer, a convolution layer, an active layer, and a second batch normalization layer, and
the basic block is driven by the quantizing of the input data, the performing of the bitwise convolution operation, and the performing of the activation function operation and the residual connection operation.
14. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 8 .
15. An electronic device, the electronic device comprising:
a host processor configured to control an operation of components comprised in the electronic device; and
a hardware accelerator configured to process input data by executing a neural network based on a command of the host processor,
wherein the hardware accelerator comprises:
a comparator;
an exclusive-NOR (XNOR) gate;
an accumulator; and
a multiplication and accumulation (MAC) operator.
16. The electronic device of claim 15 , wherein
the comparator is configured to quantize input data by comparing a result of performing a sign function operation on transformed input data with a predetermined value,
the XNOR gate and the accumulator are configured to perform a bitwise convolution operation, and
the MAC operator is configured to perform an activation function operation and a residual connection operation.
17. A device, the device comprising:
a hardware accelerator configured to, for a first neural network operation:
quantize input data by comparing a predetermined value with a result of performing a sign function operation on transformed input data, the transformed input data being determined using a first batch normalization operation;
perform a bitwise convolution operation on the quantized input data;
determine output data by applying an activation function to a result of the bitwise convolution operation;
multiply a first portion of a second batch normalization operation by the output data; and
perform an operation of adding a result of the multiplying and input data.
18. The device of claim 17 , wherein
a comparator of the hardware accelerator is configured to perform the quantizing of the input data,
an exclusive-NOR (XNOR) gate and an accumulator of the hardware accelerator are configured to perform the bitwise convolution operation, and
a multiplication and accumulation (MAC) operator of the hardware accelerator is configured to perform the determining of the output data, the multiplying of the first portion by the output data, and the performing of the operation of adding.
19. The device of claim 17 , wherein the hardware accelerator is configured to, for a second neural network operation, perform any one or any combination of any two or more of a pooling, a channel upscaling, and a third batch normalization operation based on a result of the performing of the operation of adding.