IMPROVED ARCHITECTURE FOR COUPLING DIGITAL PIXEL SENSORS AND COMPUTING COMPONENTS
The disclosed system may include a first layer that includes multiple digital pixel sensors configured to detect light. The system may also include a second layer that includes various image processing components configured to process the light detected by the digital pixel sensors. Still further, the system may include a third layer that includes machine learning (ML) hardware processing components. The image processing components of the second layer may be communicatively connected to the ML hardware processing components of the third layer via multiple micro through-silicon vias (uTSVs). Various other methods of manufacturing, apparatuses, and computer-readable media are also disclosed.
1 . A system comprising:
a first layer that includes a plurality of digital pixel sensors configured to detect light;
a second layer that includes one or more image processing components configured to process the light detected by the digital pixel sensors; and
a third layer that includes one or more machine learning (ML) hardware processing components, wherein the image processing components of the second layer are communicatively connected to the ML hardware processing components of the third layer via one or more micro through-silicon vias (uTSVs).
2 . The system of claim 1 , wherein the image processing components of the second layer comprise at least one of an analog-to-digital converter (ADC), an encoder, a memory, or a transmitter.
3 . The system of claim 1 , wherein the ML hardware processing components control capture of consecutive image frames.
4 . The system of claim 1 , wherein the ML hardware processing components are configured to execute one or more smart applications.
5 . The system of claim 4 , wherein at least one of the one or more smart applications comprises a smart image processing application.
6 . The system of claim 1 , wherein the ML hardware processing components include a plurality of processing cores.
7 . The system of claim 6 , wherein one or more of the processing cores are configured to perform local processing on a specified portion of an image captured by the plurality of digital pixel sensors.
8 . The system of claim 7 , wherein the ML processing cores share information to perform centralized processing that stitches the image captured by the plurality of digital pixel sensors together.
9 . The system of claim 6 , wherein one or more of the processing cores is configured to recognize one or more objects within a specified region of an image.
10 . The system of claim 9 , wherein one or more of the processing cores is configured to track the one or more recognized objects across a plurality of subsequent images.
11 . The system of claim 10 , wherein the one or more recognized objects are tracked across the plurality of subsequent images without neighboring pixel data from other ML processing cores.
12 . The system of claim 10 , wherein the one or more recognized objects are processed by the same ML processing core until the objects are no longer present in the image.
13 . The system of claim 10 , wherein the one or more recognized objects are processed by a plurality of ML processing cores upon determining that the objects have moved between images.
14 . An electronic device comprising:
a first layer that includes a plurality of digital pixel sensors configured to detect light;
a second layer that includes one or more image processing components configured to process the light detected by the digital pixel sensors; and
a third layer that includes one or more machine learning (ML) hardware processing components, wherein the image processing components of the second layer are communicatively connected to the ML hardware processing components of the third layer via one or more uTSVs.
15 . The electronic device of claim 14 , wherein the image processing components of the second layer are configured to combine a plurality of image bits into a combined group of image data bits.
16 . The electronic device of claim 15 , wherein the combined group of image data bits is transferred as a group to the ML hardware processing components of the third layer through the uTSVs.
17 . The electronic device of claim 15 , wherein the combined group of image data bits is transferred as a group to the ML hardware processing components using time-based multiplexing.
18 . The electronic device of claim 14 , wherein at least one of the ML hardware processing components comprises a local memory buffer.
19 . The electronic device of claim 18 , wherein image data stored in the local memory buffer is processed using a systolic array whose parameters are prestored in the local memory buffer.
20 . A method of manufacture comprising:
forming a first layer that includes a plurality of digital pixel sensors configured to detect light;
forming a second layer adhered to the first layer that includes one or more image processing components configured to process the light detected by the digital pixel sensors; and
forming a third layer adhered to the second layer that includes one or more machine learning (ML) hardware processing components, wherein the image processing components of the second layer are communicatively connected to the ML hardware processing components of the third layer via one or more uTSVs.