NPU for generating kernel of artificial neural network model and method thereof
A neural processing unit (NPU), a method for driving an artificial neural network (ANN) model, and an ANN driving apparatus are provided. The NPU includes a semiconductor circuit that includes at least one processing element (PE) configured to process an operation of an artificial neural network (ANN) model; and at least one memory configurable to store a first kernel and a first kernel filter. The NPU is configured to generate a first modulation kernel based on the first kernel and the first kernel filter and to generate second modulation kernel based on the first kernel and a second kernel filter generated by applying a mathematical function to the first kernel filter. Power consumption and memory read time are both reduced by decreasing the data size of a kernel read from a separate memory to an artificial neural network processor and/or by decreasing the number of memory read requests.
1. A neural processing unit (NPU) including a circuit, the circuit comprising:
at least one processing element (PE) configured to process an operation of an artificial neural network (ANN) model; and
at least one memory configurable to store a first kernel and a first kernel filter,
wherein the NPU is configured to generate a first modulation kernel based on the first kernel and the first kernel filter, and
wherein the first kernel filter is configured to be generated based on a difference between at least one kernel weight value of the first kernel and at least one modulation kernel weight value of the first modulation kernel.
2. The NPU of claim 1 ,
wherein the first kernel includes a K×M matrix, K and M being integers, and
wherein the K×M matrix includes at least one first weight value or weight values applicable to a first layer of the ANN model.
3. The NPU of claim 1 , wherein the first kernel filter is set during a training process of the ANN model.
4. The NPU of claim 1 , wherein the circuit is configured to generate the first modulation kernel based on the first kernel and the first kernel filter.
5. The NPU of claim 1 , wherein the circuit is configured to generate a second modulation kernel based on the first kernel and a second kernel filter.
6. The NPU of claim 5 ,
wherein the second kernel filter is set to be generated by applying a mathematical function to the first kernel filter, and
wherein the mathematical function comprises at least one of a delta function, a rotation function, a transpose function, a bias function, and a global weight function.
7. The NPU of claim 1 , wherein the circuit is configured to generate a third modulation kernel based on one among the first kernel, the first kernel filter, the mathematical function applied to the first kernel or the first kernel filter, a coefficient applied to the first kernel or the first kernel filter, and an offset applied to the first kernel or the first kernel filter.
8. The NPU of claim 1 , wherein the at least one memory is further configurable to store mapping information between at least one kernel and at least one kernel filter for generating at least one modulation kernel.
9. The NPU of claim 1 , wherein the ANN model includes information on bit allocation of first weight bits that are included in the first kernel filter for a first mode.
10. The NPU of claim 1 , wherein the NPU operates in one of a plurality of modes, the plurality of modes including:
a first mode in which a first portion of a plurality of weight bits included in the first kernel to the ANN model are applied; and
a second mode in which all of the plurality of weight bits included in the first kernel to the ANN model are applied.
11. The NPU of claim 10 , wherein the weight bits in the first portion are selected if the first portion is activated according to the first mode.
12. The NPU of claim 1 ,
wherein the first kernel includes a plurality of weight bits grouped into a first portion and a second portion, and
wherein the first portion and the second portion are configured to be used selectively.
13. The NPU of claim 1 , wherein the first kernel filter is configured such that a bit width for a value in the first kernel filter is smaller than a bit width of a weight of the first kernel.
14. An apparatus including:
a semiconductor substrate on which an electrically conductive pattern is formed;
at least one first memory electrically connected to the semiconductor substrate and configurable to store information about a first kernel; and
at least one neural processing unit (NPU) electrically connected to the substrate and configurable to access the at least one first memory, the NPU including a semiconductor circuit comprising
at least one processing element (PE) configured to process an operation of an artificial neural network (ANN) model, and
at least one internal memory configurable to store information about a first kernel filter,
wherein the operation of the ANN model includes generating a first modulation kernel based on the first kernel and the first kernel filter, and
wherein the first kernel filter is configured to be generated based on a difference between at least one kernel weight value of the first kernel and at least one modulation kernel weight value of the first modulation kernel.