Diffractive deep neural networks with differential and class-specific detection
A diffractive optical neural network device includes a plurality of diffractive substrate layers arranged in an optical path. The substrate layers are formed with physical features across surfaces thereof that collectively define a trained mapping function between an optical input and an optical output. A plurality of groups of optical sensors are configured to sense and detect the optical output, wherein each group of optical sensors has at least one optical sensor configured to capture a positive signal from the optical output and at least one optical sensor configured to capture a negative signal from the optical output. Circuitry and/or computer software receives signals or data from the optical sensors and identifies a group of optical sensors in which a normalized differential signal calculated from the positive and negative optical sensors within each group is the largest or the smallest of among all the groups.
1. A diffractive optical neural network device for machine learning, classification, and/or processing of at least one optical image, signal, or data comprising:
a plurality of optically transmissive and/or reflective substrate layers arranged in one or more optical paths, each of the plurality of optically transmissive and/or reflective substrate layers comprising a plurality of physical features formed on or within the optically transmissive and/or reflective substrate layers and having different complex-valued transmission and/or reflection coefficients as a function of lateral coordinates across each substrate layer, wherein the plurality of optically transmissive and/or reflective substrate layers and the plurality of physical features thereon collectively define a trained mapping function between an input optical image, input optical signal, or input data to the plurality of optically transmissive and/or reflective substrate layers and one or more output optical images, output optical signals, or data created by optical diffraction/reflection through/off the plurality of optically transmissive and/or reflective substrate layers;
a plurality of groups of optical sensors configured to sense and detect the output optical images, output optical signals, or data resulting from the plurality of optically transmissive and/or reflective substrate layers, wherein each group of optical sensors comprises at least one optical sensor configured to capture a positive signal from the output optical images, output optical signals, or data and at least one optical sensor configured to capture a negative signal from the output optical images, output optical signals, or data; and
circuitry and/or computer software configured to identify a group of optical sensors within the plurality of groups of optical sensors in which a normalized signal difference calculated from the positive and negative optical sensors within each group that has a largest or a smallest normalized signal difference of among all the groups.
2. The diffractive optical neural network device of claim 1 , wherein the plurality of groups of optical sensors are assigned to represent different classes of the input optical image, input optical signal, or input data, and the group with the largest or the smallest normalized signal difference calculated from the positive and negative optical sensors determine an inference for the class of input optical image, input optical signal, or input data.
3. The diffractive optical neural network device of claim 1 , wherein the signals from the respective optical sensors are multiplied by trainable coefficients or scaling factors.
4. The diffractive optical neural network device of claim 1 , wherein the plurality of groups of optical sensors are located on a common plane or surface.
5. The diffractive optical neural network device of claim 1 , wherein at least some of the groups of optical sensors are located on different planes or surfaces.
6. The diffractive optical neural network device of claim 1 , wherein one or more of the optically transmissive and/or reflective substrate layers comprise reconfigurable spatial light modulators.
7. A diffractive optical neural network device for machine learning, classification, and/or processing of at least one optical image, signal, or data comprising:
a plurality of diffractive optical neural network devices configured to receive an input optical image, input optical signal, or data, each of the plurality of diffractive optical neural network devices comprising:
a plurality of optically transmissive and/or reflective substrate layers arranged in one or more optical paths, each of the optically transmissive and/or reflective substrate layers comprising a plurality of physical features formed on or within the plurality of optically transmissive and/or reflective substrate layers and having different complex-valued transmission and/or reflection coefficients as a function of lateral coordinates across each substrate layer, wherein the plurality of optically transmissive and/or reflective substrate layers and the plurality of physical features thereon collectively define a trained mapping function between the input optical image, input optical signal, or data to the plurality of optically transmissive and/or reflective substrate layers and one or more output optical images, output optical signals, or data created by optical diffraction/reflection through/off the plurality of optically transmissive and/or reflective substrate layers;
one or more optical sensors configured to sense and detect the output optical images, output optical signals, or data resulting from the plurality of optically transmissive and/or reflective substrate layers for each of the plurality of diffractive optical neural network devices.
8. The diffractive optical neural network device of claim 7 , wherein the one or more optical sensors are located on a common plane or surface.
9. The diffractive optical neural network device of claim 7 , wherein the one or more optical sensors are located on different planes or surfaces.
10. The diffractive optical neural network device of claim 7 , wherein the one or more optical sensors comprise a plurality of groups of optical sensors configured to sense the output optical images, output optical signals, or data resulting from each of the diffractive optical neural network devices, wherein each group of optical sensors comprises at least one optical sensor configured to capture a positive signal from the output optical images, output optical signals, or data and at least one optical sensor configured to capture a negative signal from the output optical images, output optical signals, or data.
11. The diffractive optical neural network device of claim 10 , further comprising circuitry and/or computer software configured to identify a group of optical sensors within the plurality of groups of optical sensors in which a normalized signal difference between the optical sensor groups that has a largest or a smallest normalized signal difference of the plurality of groups of optical sensors, and determining a class of the input optical image, input optical signal, or data.
12. The diffractive optical neural network device of claim 10 , wherein the signals from the respective optical sensors are multiplied by trainable coefficients or scaling factors.
13. The diffractive optical neural network device of claim 7 , wherein one or more of the optically transmissive and/or reflective substrate layers comprise reconfigurable spatial light modulators.
14. A diffractive optical neural network device for machine learning, classifying, and/or processing at least one optical image, signal, or data comprising:
a plurality of optically transmissive and/or reflective substrate layers arranged in one or more optical paths, each of plurality of optically transmissive and/or reflective substrate layers comprising a plurality of physical features formed on or within plurality of optically transmissive and/or reflective substrate layers and having different complex-valued transmission and/or reflection coefficients as a function of lateral coordinates across each substrate layer, wherein the plurality of optically transmissive and/or reflective substrate layers and the plurality of physical features thereon collectively define a trained mapping function between an input optical image, optical signal, or data to plurality of optically transmissive and/or reflective substrate layers and one or more output optical images, optical signals, or data created by optical diffraction/reflection through/off the plurality of optically transmissive and/or reflective substrate layers;
a plurality of groups of optical sensors configured to sense and detect the output optical images, optical signals, or data resulting from plurality of optically transmissive and/or reflective substrate layers, wherein each group of optical sensors comprises at least one optical sensor configured to capture a positive signal from the one or more output optical images, optical signals, or data and at least one optical sensor configured to capture a negative signal from the one or more output optical images, optical signals, or data, wherein the optical sensors of each group are each associated with individual scaling factors; and
circuitry and/or computer software configured to identify a group of optical sensors within the plurality of groups of optical sensors in which a signal difference between the optical sensor groups calculated using the corresponding scaling factors that has a largest or a smallest normalized signal difference of the plurality of groups of optical sensors.
15. The device of claim 14 , wherein the scaling factors are pre-determined.
16. The device of claim 14 , wherein the scaling factors are learned using machine learning and training.
17. The device of claim 14 , wherein the scaling factors are all equal to 1 or another constant for each class of image, signal, or data.
18. The device of claim 14 , wherein one or more of the optically transmissive and/or reflective substrate layers comprise reconfigurable spatial light modulators.
19. A diffractive optical neural network device for machine learning, classifying, and/or processing at least one optical image, signal, or data comprising:
a plurality of diffractive optical neural network devices configured to receive light from an input optical image, signal, or data, each of the plurality of diffractive optical neural network devices comprising:
a plurality of optically transmissive and/or reflective substrate layers arranged in one or more optical paths, each of the plurality of optically transmissive and/or reflective substrate layers comprising a plurality of physical features formed on or within the plurality of optically transmissive and/or reflective substrate layers and having different complex-valued transmission/reflection coefficients as a function of lateral coordinates across each substrate layer, wherein the plurality of optically transmissive and/or reflective substrate layers and the plurality of physical features thereon collectively define a trained mapping function between an input optical image, optical signal, or data to the plurality of optically transmissive and/or reflective substrate layers and one or more output optical images, optical signals, or data created by optical diffraction/reflection through/off the plurality of optically transmissive and/or reflective substrate layers;
a plurality of groups of optical sensors configured to sense and detect the output optical images, optical signals, or data resulting from the plurality of optically transmissive and/or reflective substrate layers, wherein each group of optical sensors comprises at least one optical sensor configured to capture a positive signal from the one or more output optical images, optical signals, or data and at least one optical sensor configured to capture a negative signal from the one or more output optical images, optical signals, or data, wherein the optical sensors of each group are each associated with individual scaling factors; and
circuitry and/or computer software configured to identify a group of optical sensors within the plurality of groups of optical sensors in which a signal difference between the optical sensor groups calculated using the corresponding scaling factors that has a largest or a smallest normalized signal difference of the plurality of groups of optical sensors, wherein the identified group corresponds to a particular class of the input optical image, signal, or data.
20. The device of claim 19 , wherein the scaling factors are pre-determined.
21. The device of claim 19 , wherein the scaling factors are learned using machine learning and training.
22. The device of claim 19 , wherein the scaling factors are all equal to 1 or another constant for each class of image, signal, or data.
23. The device of claim 19 , wherein one or more of the optically transmissive and/or reflective substrate layers comprise reconfigurable spatial light modulators.