Integrated sensing and machine learning processing devices
The present disclosure provides for a semiconductor device with integrated sensing and processing functionalities. The semiconductor device includes a sensing module configured to generate a plurality of analog sensing signals; and a machine learning (ML) processor. The sensing module and the ML processor are fabricated on a single wafer. The ML processor includes crossbar arrays that processes the analog sensing signals to generate analog preprocessed sensing data; an analog-to-digital converter (ADC) to convert the analog preprocessed sensing data into digital preprocessed sensing data; and a machine learning processing unit to process the digital preprocessed sensing data utilizing one or more machine learning model.
1. A semiconductor device, comprising:
a sensing module configured to generate a plurality of analog sensing signals; and
a machine learning (ML) processor, comprising:
one or more crossbar arrays configured to process the analog sensing signals to generate analog preprocessed sensing data;
an analog-to-digital converter (ADC) configured to convert the analog preprocessed sensing data into digital preprocessed sensing data; and
a machine learning processing unit configured to process the digital preprocessed sensing data utilizing one or more machine learning models, wherein the sensing module and the ML processor are fabricated on a single wafer, wherein the sensing module is fabricated on a first portion of the wafer, and wherein the ML processor is fabricated on a second portion of the wafer that surrounds the first portion of the wafer.
2. The semiconductor device of claim 1 , wherein the sensing module and the ML processor are fabricated on the same side of the wafer.
3. The semiconductor device of claim 2 , wherein a plurality of transistors of the ML processor and a plurality of sensors of the sensing module are fabricated on a substrate of the wafer, and wherein the semiconductor device further comprises a plurality of interconnect layers fabricated on the plurality of transistors and the plurality of sensors.
4. The semiconductor device of claim 3 , wherein the plurality of sensors comprises a plurality of photodiodes, wherein a first metallic via, a second metallic via, and a third metallic via of a first interconnect layer are connected to a source region, a gate region, and a drain region of a first transistor of the plurality of transistors, respectively, and wherein a fourth metallic via and a fifth metallic via of the first interconnect layer are connected to a first photodiode of the plurality of photodiodes.
5. The semiconductor device of claim 4 , wherein a resistive random-access memory (RRAM) device of the ML processor is fabricated on a metallic pad or metallic via of an interconnect layer of the plurality of interconnect layers, and wherein the RRAM device is connected to the first transistor through the first interconnect layer.
6. The semiconductor device of claim 5 , wherein a micro lens and a set of color filters of the sensing module are fabricated on the plurality of interconnect layers.
7. The semiconductor device of claim 2 , wherein the sensing module is fabricated on the ML processor.
8. The semiconductor device of claim 1 , wherein the sensing module and the ML processor are fabricated on different sides of the wafer.
9. The semiconductor device of claim 1 , wherein the sensing module comprises an array of image sensors, wherein the plurality of analog sensing signals comprises a plurality of analog image signals.
10. The semiconductor device of claim 1 , wherein the analog preprocessed sensing data correspond to a plurality of features extracted from the analog sensing signals, and wherein the machine learning processing unit performs machine learning using the extracted features.
11. The semiconductor device of claim 1 , further comprising a packaging substrate, wherein the wafer is connected to the packaging substrate through an interconnect layer.
12. The semiconductor device of claim 1 , wherein the ML processor is powered utilizing the analog sensing signals.
13. The semiconductor device of claim 1 , further comprising a transceiver configured to:
transmit, to a computing device, a predictive output generated by the machine learning processing unit based on the one or more machine learning models; and
receive, from the computing device, instructions for performing operations based on the predictive output.
14. The semiconductor device of claim 1 , wherein the analog preprocessed sensing data represents a convolution of the analog sensing signals and a kernel.
15. The semiconductor device of claim 14 , wherein conductance values of a plurality of cross-point devices of the one or more crossbar arrays are programmed to values representing the kernel.
16. The semiconductor device of claim 1 , wherein the sensing module comprises a two-dimensional sensor array, wherein a plurality of cross-point devices of the one or more crossbar arrays is configured to receive the analog sensing signals produced by the two-dimensional sensor array as input.
17. The semiconductor device of claim 16 , wherein the one or more crossbar arrays comprises a plurality of crossbar arrays positioned on a plurality of different planes.
18. A semiconductor device, comprising:
a sensing module configured to generate a plurality of analog sensing signals; and
a machine learning processor configured to produce a predictive output by processing the plurality of analog sensing signals using one or more machine learning models, wherein the machine learning processor comprises:
a plurality of crossbar arrays configured to generate a plurality of analog outputs representative of the predictive output; and
an analog-to-digital converter unit configured to convert the plurality of analog outputs representative of the predictive output into a plurality of digital signals representative of the predictive output, wherein the sensing module and the machine learning processor are fabricated on a wafer, wherein the sensing module is fabricated on a first portion of the wafer, and wherein the machine learning processor is fabricated on a second portion of the wafer that surrounds the first portion of the wafer.
19. The semiconductor device of claim 18 , further comprising a packaging substrate, wherein the wafer is connected to the packaging substrate through an interconnect layer.