Face recognition using depth and multi-spectral camera
A camera is configured to output a test depth+multi-spectral image including a plurality of pixels. Each pixel corresponds to one of the plurality of sensors of a sensor array of the camera and includes at least a depth value and a spectral value for each spectral light sub-band of a plurality of spectral illuminators of the camera. A face recognition machine is previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image. The face recognition machine is configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a face.
1. A camera comprising:
a sensor array including a plurality of sensors;
an infrared (IR) illuminator configured to emit active IR light in an IR light sub-band;
a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band;
a depth controller machine configured to determine a depth value for each of the plurality of sensors based on the active IR light,
a spectral controller machine configured to, for each of the plurality of sensors, determine a spectral value for each spectral light sub-band of the plurality of spectral illuminators; and
an output machine configured to output a test depth+multi-spectral image including a plurality of pixels, each pixel corresponding to one of the plurality of sensors of the sensor array and including at least:
a depth value, and
a spectral value for each spectral light sub-band of the plurality of spectral illuminators;
a face recognition machine previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image, the face recognition machine configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a face.
2. The camera of claim 1 , wherein each spectral value is calculated based on the depth value determined for the sensor that corresponds to the pixel.
3. The camera of claim 1 , wherein the face recognition machine is configured to use a convolutional neural network to determine the confidence value.
4. The camera of claim 3 , wherein the face recognition machine includes a plurality of input nodes, wherein each input node is configured to receive a pixel value array corresponding to a different pixel of the plurality of pixels of the test depth+multi-spectral image, and wherein the pixel value array includes the depth value and the plurality of multi-spectral values for the pixel.
5. The camera of claim 4 , wherein the plurality of multi-spectral values for the pixel include more than three spectral values.
6. The camera of claim 4 , wherein the output machine is configured to output a surface normal for each pixel of the test depth+multi-spectral image, and wherein the pixel value array includes the surface normal.
7. The camera of claim 4 , wherein the output machine is configured to output a curvature for each pixel of the test depth+multi-spectral image, and wherein the pixel value array includes the curvature.
8. The camera of claim 3 , wherein the face recognition machine is configured to use a plurality of models to determine the confidence value, wherein the plurality of models includes a plurality of channel-specific models, wherein each channel-specific model is configured to process a different pixel parameter for the plurality of pixels of the test depth+multi-spectral image, wherein each channel-specific model includes a plurality of input nodes, and wherein, for each channel-specific model, each input node is configured to receive a pixel parameter value for a different pixel of the plurality of pixels of the test depth+multi-spectral image.
9. The camera of claim 1 , wherein the face recognition machine is configured to use a statistical model to determine the confidence value.
10. The camera of claim 9 , wherein the statistical model includes a nearest neighbor algorithm.
11. The camera of claim 9 , wherein the statistical model includes a support vector machine.
12. The camera of claim 1 , wherein the face recognition machine is further configured to output a location on the test depth+multi-spectral image of a bounding box around a recognized face.
13. The camera of claim 1 , wherein the face recognition machine is further configured to output a location on the test depth+multi-spectral image of an identified two-dimensional (2D) facial feature of a recognized face.
14. The camera of claim 1 , wherein the face recognition machine is further configured to output a location on the test depth+multi-spectral image of an identified three-dimensional (3D) facial feature of a recognized face.
15. The camera of claim 1 , wherein the face recognition machine is further configured to output a location on the test depth+multi-spectral image of an identified spectral feature on a recognized face.
16. The camera of claim 1 , wherein the face recognition machine is further configured to output, for each pixel of the test depth+multi-spectral image, a confidence value indicating a likelihood that the pixel is included in a face.
17. The camera of claim 1 , wherein the face recognition machine is further configured to output an identity of a face recognized in the test depth+multi-spectral image.
18. The camera of claim 1 , wherein the plurality of sensors of the sensor array are differential sensors, and wherein each spectral value is determined based on a depth value and a differential measurement for that differential sensor.
19. A camera comprising:
a sensor array including a plurality of sensors;
an infrared (IR) illuminator configured to emit active IR light in an IR light sub-band;
a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band;
a depth controller machine configured to determine a depth value for each of the plurality of sensors based on the active IR light,
a spectral controller machine configured to, for each of the plurality of sensors, determine a spectral value for each spectral light sub-band of the plurality of spectral illuminators, wherein each spectral value is calculated based on the depth value determined for the sensor that corresponds to the pixel; and
an output machine configured to output a test depth+multi-spectral image including a plurality of pixels, each pixel corresponding to one of the plurality of sensors of the sensor array and including at least:
a depth value, and
a spectral value for each spectral light sub-band of the plurality of spectral illuminators; and
a face recognition machine including a convolutional neural network previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image, the face recognition machine configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a face.