Reduced power consumption analog or hybrid mac neural network
Power efficient performance may be implemented in a hardware accelerator (e.g., a neural processor) comprising hybrid or analog multiply and accumulate (MAC) processing elements (PEs). For example, power consumption may be reduced in neural networks with a rectified linear unit (ReLU) activation layer. A hybrid or analog MAC circuit may be configured with a look-ahead sign detector to dynamically stop computations prior to completion, for example, based on detection of a negative value, which a ReLU activation layer may (e.g., subsequently) convert to zero. The sign of a value may be indicated by a most significant bit (MSB). A controller may provide power and/or clock cycles to an analog to digital converter (ADC) to determine a sign of a value being computed. The sign may be used to selectively complete computations for positive values and selectively terminate computations for negative values, thereby reducing power consumption of the MAC circuit.
1 . A neural processing unit (NPU) configured to implement an artificial intelligence (AI) neural network (NN) model, the NPU comprising:
a multiply and accumulate (MAC) circuit comprising:
a plurality of MAC processing elements (PEs) configured to perform multiplication operations on inputs and weights to generate mid-term values and to accumulate the mid-term values as an analog output;
an analog to digital converter (ADC) configured to convert the analog output to a digital output; and
an ADC controller configured to stop operation of the ADC based at least on an indication of a type of activation function applied to the digital output and a sign of the digital output, thereby reducing power consumption of the MAC circuit.
2 . The NPU of claim 1 , wherein the MAC circuit comprises an analog MAC circuit or a hybrid MAC circuit.
3 . The NPU of claim 1 , wherein the ADC controller is configured to stop operation of the ADC by interrupting a power supply to the ADC.
4 . The NPU of claim 1 , wherein the ADC comprises a successive approximation register (SAR) ADC and wherein the ADC controller is configured to stop operation of the SAR ADC to reduce a number of cycles of the SAR ADC.
5 . The NPU of claim 1 , wherein the ADC controller is configured to selectively stop the operation of the ADC.
6 . The NPU of claim 1 , wherein the ADC controller is configured to selectively stop the operation of the ADC based on an indication that the activation function applied to the digital output comprises a rectified linear unit (ReLU) activation function and the sign of the digital output is negative.
7 . The NPU of claim 1 , the MAC circuit further comprising: an activation function detector configured to determine the type of activation function.
8 . The NPU of claim 1 , the MAC circuit further comprising:
a look-ahead sign detector configured to determine the sign of the digital output.
9 . A method of improving implementation of an artificial intelligence (AI) neural network (NN) model, the method comprising:
performing multiply and accumulate (MAC) operations on inputs and weights provided to a MAC circuit;
generating an analog output from the MAC operations;
partially converting, by an analog to digital converter (ADC), the analog output to a digital output;
determining that a sign of the digital output based on the partial conversion; is a first sign or a second sign, wherein the second sign is different from the first sign;
completing conversion of the analog output to the digital output by the ADC responsive to at least the sign being the first sign; and
reducing power consumption of the MAC circuit by stopping conversion of the analog output to the digital output by the ADC based at least on an indication of a first type of activation function applied to the digital output and responsive to at least the sign being the second sign.
10 . The method of claim 9 , wherein the partially converting comprises converting the analog output to a most significant bit (MSB) of the digital output, and wherein the sign of the digital output is indicated by the MSB.
11 . The method of claim 9 , wherein the ADC comprises a successive approximation register (SAR) ADC, and wherein said interrupting the power supply reduces the power consumption of the MAC circuit by reducing a number of cycles of the SAR ADC.
12 . The method of claim 9 , wherein said stopping the conversion of the analog output to the digital output by the ADC includes selectively stopping operation of the ADC.
13 . The method of claim 9 , wherein the first type of activation function is a rectified linear unit (ReLU) activation function, the second sign of the digital output is negative and the first sign of the digital output is positive.
14 . The method of claim 9 , further comprising: determining the type of activation function to be one of the first type of activation function or a second type of activation function.
15 . A system, comprising:
one or more memory devices that store a deep neural network (DNN) topology and a neural processing unit (NPU) comprising:
a multiply and accumulate (MAC) circuit comprising:
a plurality of MAC processing elements (PEs) configured to generate an analog output from multiply and accumulation operations on inputs and weights;
an analog to digital converter (ADC) configured to convert the analog output to a digital output; and
an ADC controller configured to selectively stop the ADC before completely converting the analog output to the digital output based at least on an indication of a type of activation function applied to the digital output and a sign of the digital output, thereby reducing power consumption of the MAC circuit.
16 . The system of claim 15 , wherein the ADC controller is configured to selectively stop the ADC based on an indication that the activation function comprises a rectified linear unit (ReLU) activation function and the sign of the digital output is negative.
17 . The system of claim 15 , the MAC circuit further comprising:
an activation function detector configured to determine the type of activation function.
18 . The system of claim 15 , the MAC circuit further comprising:
a look-ahead sign detector configured to determine the sign of the digital output.
19 . The NPU of claim 1 , wherein to stop operation of the ADC before completely converting the analog output to the digital output, the ADC controller is further configured to:
interrupt a clock provided to the ADC, thereby reducing a number of clock cycles during which the ADC performs conversion to reduce ADC energy consumption.
20 . The system of claim 15 , wherein to selectively stop the ADC before completely converting the analog output to the digital output, the ADC controller is further configured to:
interrupt a clock provided to the ADC, thereby reducing a number of clock cycles during which the ADC performs conversion to reduce ADC energy consumption.