IMAGING SYSTEMS AND METHODS FOR PERFORMING ANALOG DOMAIN REGIONAL PIXEL LEVEL FEATURE EXTRACTION
Imaging circuitry may include circuits for implementing charge mode feature extraction in the analog domain. The imaging circuitry may include pixels configured to generate pixel values. The pixel values may then be weighted using adjustable weighting circuits to generate corresponding weighted pixel values. The weighted pixels values may then be combined to obtain an output neuron voltage for at least one layer in a neural network. The output neuron voltage may be stored in idle pixels, may be combined with other weighted pixel values, and may be otherwise manipulated prior to being processed in the digital domain. Performing feature extraction in the analog domain for each layer of results in the neural network saves power and area by avoiding the need to move data around to conventional digital memories.
1 . Imaging circuitry, comprising:
a first pixel configured to output a first pixel value;
a second pixel configured to output a second pixel value;
a first adjustable circuit configured to apply a first weighting factor to the first pixel value to generate a first weighted pixel value;
a second adjustable circuit configured to apply a second weighting factor to the second pixel value to generate a second weighted pixel value; and
an output circuit configured to combine the first weighted pixel value and the second weighted pixel value to generate an analog output voltage.
2 . The imaging circuitry of claim 1 , further comprising analog circuitry configured to store the analog output voltage in the analog domain.
3 . The imaging circuitry of claim 1 , wherein the first and second pixels are formed on a first die, and wherein the first and second adjustable circuits and the output circuit are formed on a second die stacked under the first die.
4 . The imaging circuitry of claim 3 , wherein the second die comprises:
local output buses configured to route the first and second pixel values to peripheral circuits on the second die.
5 . The imaging circuitry of claim 4 , wherein the second die comprises:
configurable buses that route source follower drain terminals in the first and second pixels to the peripheral circuits to generate and sum the first and second weighted values.
6 . The imaging circuitry of claim 4 , wherein the second die further comprises:
additional local output buses configured to support parallel generation and summing of the first and second weighted values.
7 . The imaging circuitry of claim 3 , wherein the first and second pixels are part of an array of image sensor pixels on the first die, and wherein the first pixel value is coupled to the second die via a global output bus that is configured to receive pixel values from other pixels in the array.
8 . The imaging circuitry of claim 3 , wherein the first and second pixels are part of an array of image sensor pixels, and wherein the first pixel value is coupled to the second die via a global output bus that is configured to receive pixel values from other pixels in only a subset of the array.
9 . The imaging circuitry of claim 1 , where the first pixel value is sensed multiple times at different weight levels by using a separate optically black reference pixel.
10 . The imaging circuitry of claim 1 , wherein the first pixel value is not used for subsequent readout through an analog-to-digital converter but is used only to combine with additional weighted pixel values.
11 . The imaging circuitry of claim 1 , wherein the first and second adjustable circuits comprise adjustable capacitor circuits.
12 . The imaging circuitry of claim 11 , wherein the output circuit comprises at least one output capacitor.
13 . The imaging circuitry of claim 11 , wherein the output circuit comprises a positive output capacitor configured to store charge associated with positive weighting factors and a second output capacitor configured to store charge associated with negative weighting factors.
14 . The imaging circuitry of claim 1 , wherein the first and second adjustable circuits comprise adjustable resistor circuits.
15 . The imaging circuitry of claim 1 , wherein the first and second adjustable circuits comprise adjustable current mirroring circuits.
16 . The imaging circuitry of claim 1 , wherein the first and second adjustable circuits comprise resistive non-volatile memory.
17 . Imaging circuitry, comprising:
a first pixel having a first source follower drain terminal, wherein the first pixel is configured to output a first pixel value;
a second pixel having a second source follower drain terminal, wherein the second pixel is configured to output a second pixel value; and
current mirror circuitry having a first set of adjustable switches for applying a first weight to the first pixel value and a second set of adjustable switches for applying a second weight to the second pixel value.
18 . The imaging circuitry of claim 17 , wherein the first set of adjustable switches comprises switches of different sizes.
19 . The imaging circuitry of claim 17 , further comprising:
a switch capacitor based integrating circuit configured to receive signals from the current mirror circuitry.
20 . Imaging circuitry, comprising:
a first group of active pixels configured to generate active pixel values;
weighting circuits configured to receive the active pixel values from the first group of active pixels and to generate corresponding weighted pixel values;
an output circuit configured to receive and combine the weighted pixel values to generate corresponding output voltages; and
a second group of idle pixels configured to temporarily store the output voltages to avoid having to store the output voltages in the digital domain.