Object recognition using depth and multi-spectral camera

1. A camera comprising:

a sensor array including a plurality of sensors;

an infrared (IR) illuminator configured to emit active IR light in an IR light sub-band;

a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band;

a depth controller machine configured to determine a depth value for each of the plurality of sensors based on the active IR light,

a spectral controller machine configured to, for each of the plurality of sensors, determine a spectral value for each spectral light sub-band of the plurality of spectral illuminators; and

an output machine configured to output a test depth+multi-spectral image including a plurality of pixels, each pixel corresponding to one of the plurality of sensors of the sensor array and including at least:

a depth value, and

a spectral value for each spectral light sub-band of the plurality of spectral illuminators;

an object recognition machine previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image, the object recognition machine configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a specified object.

2. The camera of claim 1 , wherein the object recognition machine is further configured to output a confidence value corresponding to each of a plurality of different types of objects, each confidence value indicating the likelihood that the test depth+multi-spectral image includes that type of object.

3. The camera of claim 2 , wherein the object recognition machine is further configured to output a location on the test depth+multi-spectral image of each recognized object.

4. The camera of claim 2 , wherein the object recognition machine is further configured to output, for each pixel of the test depth+multi-spectral image, a plurality of confidence values corresponding to the plurality of different types of objects, each confidence value indicating a likelihood that the pixel is included in that type of object.

5. The camera of claim 1 , wherein each spectral value is determined based on the depth value determined for the sensor that corresponds to the pixel.

6. The camera of claim 1 , wherein the object recognition machine is configured to use a convolutional neural network to determine the confidence value.

7. The camera of claim 6 , wherein the object recognition machine includes a plurality of input nodes, wherein each input node is configured to receive a pixel value array corresponding to a different pixel of the plurality of pixels of the test depth+multi-spectral image, and wherein the pixel value array includes the depth value and the plurality of multi-spectral values for the pixel.

8. The camera of claim 7 , wherein the plurality of multi-spectral values for the pixel include more than three spectral values.

9. The camera of claim 7 , wherein the output machine is configured to output a surface normal for each pixel of the test depth+multi-spectral image, and wherein the pixel value array includes the surface normal.

10. The camera of claim 7 , wherein the output machine is configured to output a curvature for each pixel of the test depth+multi-spectral image, and wherein the pixel value array includes the curvature.

11. The camera of claim 6 , wherein the object recognition machine is configured to use a plurality of models to determine the confidence value, wherein the plurality of models includes a plurality of channel-specific models, wherein each channel-specific model is configured to process a different pixel parameter for the plurality of pixels of the test depth+multi-spectral image, wherein each channel-specific model includes a plurality of input nodes, and wherein, for each channel-specific model, each input node is configured to receive a pixel parameter value for a different pixel of the plurality of pixels of the test depth+multi-spectral image.

12. The camera of claim 1 , wherein the object recognition machine is configured to use a statistical model to determine the confidence value.

13. The camera of claim 12 , wherein the statistical model includes a nearest neighbor algorithm.

14. The camera of claim 12 , wherein the statistical model includes a support vector machine.

15. The camera of claim 1 , wherein the object recognition machine is further configured to output a location on the test depth+multi-spectral image of a bounding box around the specified object.

16. The camera of claim 1 , wherein the object recognition machine is further configured to output, for each pixel of the test depth+multi-spectral image, a confidence value indicating a likelihood that the pixel is included in the specified object.

17. The camera of claim 1 , wherein the object recognition machine is further configured to output one or more of a location on the test depth+multi-spectral image of an identified two-dimensional (2D) feature of the specified object, a location on the test depth+multi-spectral image of an identified three-dimensional (3D) feature of the specified object, and a location on the test depth+multi-spectral image of an identified spectral feature of the specified object.

18. The camera of claim 1 , wherein the plurality of sensors of the sensor array are differential sensors, and wherein each spectral value is determined based on a depth value and a differential measurement for that differential sensor.

19. A method comprising:

actively illuminating an object with active IR light;

while the object is actively illuminated with active IR light, measuring depth contours of the object with a plurality of sensors of a camera;

actively illuminating the object with active spectral light in each of a plurality of different spectral light sub-bands;

while the object is actively illuminated with active spectral light in each of the plurality of different spectral light sub-bands, measuring a spectral signature of the object in the spectral light sub-band with the plurality of sensors of the camera used to measure the depth contours of the object; and

computer recognizing the object based on the measured depth contours and the measured spectral signatures.

20. A camera comprising:

a sensor array including a plurality of sensors;

an infrared (IR) illuminator configured to emit active IR light in an IR light sub-band;

a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band;

a depth controller machine configured to determine a depth value for each of the plurality of sensors based on the active IR light,

a spectral controller machine configured to, for each of the plurality of sensors, determine a spectral value for each spectral light sub-band of the plurality of spectral illuminators, wherein each spectral value is calculated based on the depth value determined for the sensor that corresponds to the pixel; and

an output machine configured to output a test depth+multi-spectral image including a plurality of pixels, each pixel corresponding to one of the plurality of sensors of the sensor array and including at least:

a depth value, and

a spectral value for each spectral light sub-band of the plurality of spectral illuminators; and

an object recognition machine including a convolutional neural network previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image, the object recognition machine configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a specified object.