Depth sensor calibration and per-pixel correction

1. A method of correcting an input depth image, comprising:

receiving the input depth image captured by a depth sensor, wherein the input depth image comprises pixels, the pixels having respective depth values in the input depth image;

determining per-pixel correction values for the pixels utilizing depth calibration data for a non-linear error model calibrated for the depth sensor, wherein the per-pixel correction values are determined based on portions of the depth calibration data respectively corresponding to the pixels and the depth values, and wherein the depth calibration data for the non-linear error model calibrated for the depth sensor is generated directly based on depth error values for the pixels and ground truth measurements;

applying the per-pixel correction values to the depth values to generate a corrected depth image; and

outputting the corrected depth image.

2. The method of claim 1 , wherein the depth calibration data for the non-linear error model calibrated for the depth sensor comprises a parameter matrix, and wherein the parameter matrix comprises error fitting parameters for the pixels.

3. The method of claim 2 , wherein determining the per-pixel correction values based on the portions of the depth calibration data respectively corresponding to the pixels and the depth values further comprises:

for each of the pixels of the input depth image:

retrieving error fitting parameters for a pixel from the parameter matrix, wherein the pixel has a depth value in the input depth image; and

computing a per-pixel correction value for the pixel by evaluating a non-linear error function using the error fitting parameters for the pixel, wherein the non-linear error function is evaluated for the input depth image based on the depth value.

4. The method of claim 2 , wherein the input depth image comprises a plurality of patches that comprise respective pluralities of the pixels of the input depth image, wherein the patches are assigned respective error fitting parameters of the parameter matrix, and wherein a plurality of the pixels included in a patch shares error fitting parameters assigned to the patch.

5. The method of claim 1 , wherein the depth calibration data for the non-linear error model calibrated for the depth sensor comprises a lookup table, and wherein the lookup table comprises predefined correction values at preset depth values for the pixels.

6. The method of claim 5 , wherein determining the per-pixel correction values based on the portions of the depth calibration data respectively corresponding to the pixels and the depth values further comprises:

for each of the pixels of the input depth image:

retrieving a first predefined correction value at a first preset depth value for a pixel from the lookup table, wherein the pixel has a depth value in the input depth image, and wherein the first preset depth value is equal to or immediately greater than the depth value for the pixel in the lookup table;

retrieving a second predefined correction value at a second preset depth value for the pixel from the lookup table, wherein the second preset depth value is immediately less than the depth value for the pixel in the lookup table; and

determining a per-pixel correction value for the pixel having the depth value based upon linear interpolation between the first predefined correction value at the first preset depth value and the second predefined correction value at the second preset depth value.

7. The method of claim 1 , further comprising:

identifying pluralities of the pixels of the input depth image that form respective patches, wherein the patches are assigned respective subsets of the depth calibration data, and wherein a plurality of the pixels included in a patch shares a subset of the depth calibration data.

8. The method of claim 7 , wherein each of the patches is a preset fraction of the input depth image, the preset fraction of the input depth image being independent of a resolution of the input depth image captured by the depth sensor.

9. The method of claim 7 , further comprising:

responsive to applying the per-pixel correction values to the depth values, filtering the corrected depth image to smooth between the patches, wherein the corrected depth image as filtered is outputted.

10. The method of claim 1 , wherein the corrected depth image is generated by the depth sensor.

11. The method of claim 1 , further comprising generating the depth calibration data for the depth sensor from depth images of a surface captured by the depth sensor and corresponding ground truth measurements to the surface measured by a distance measurement device.

12. A system that corrects an input depth image, comprising:

at least one processor; and

memory that comprises computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform acts including:

receiving the input depth image captured by a depth sensor, wherein the input depth image comprises pixels, the pixels having respective depth values in the input depth image;

determining per-pixel correction values for the pixels utilizing depth calibration data for a non-linear error model calibrated for the depth sensor, wherein determining the per-pixel correction values for the pixels further comprises:

identifying pluralities of the pixels of the input depth image that form respective patches;

wherein the patches are assigned respective subsets of the depth calibration data;

wherein a plurality of the pixels included in a patch shares a subset of the depth calibration data;

wherein the per-pixel correction values are determined based on the subsets of the depth calibration data respectively corresponding to the pixels and the depth values; and

wherein the depth calibration data for the non-linear error model calibrated for the depth sensor is generated directly based on depth error values for the pixels and ground truth measurements;

applying the per-pixel correction values to the depth values to generate a corrected depth image; and

outputting the corrected depth image.

13. The system of claim 12 , wherein the depth calibration data for the non-linear error model calibrated for the depth sensor comprises a parameter matrix, and wherein the parameter matrix comprises error fitting parameters for the pixels.

14. The system of claim 12 , wherein the depth calibration data for the non-linear error model calibrated for the depth sensor comprises a lookup table, and wherein the lookup table comprises predefined correction values at preset depth values for the pixels.

15. The system of claim 12 , wherein each of the patches is a preset fraction of the input depth image, the preset fraction of the input depth image being independent of a resolution of the input depth image captured by the depth sensor.

16. The system of claim 12 , the memory further comprises computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform acts including:

filtering the corrected depth image to smooth between the patches.

17. A method of calibrating a depth sensor, comprising:

receiving depth images of a surface captured by the depth sensor, subsets of the depth images of the surface being captured at respective distances from the surface, the depth images each comprise pixels, the pixels having respective depth values in each of the depth images;

receiving respective ground truth measurements to the surface corresponding to the subsets of the depth images, the ground truth measurements being measured by a distance measurement device;

averaging the depth values of the pixels over the depth images in each of the subsets captured at the respective distances from the surface;

comparing averaged depth values of the pixels to the ground truth measurements to compute error values for the pixels at each of the respective distances from the surface; and

generating depth calibration data for a non-linear error model calibrated for the depth sensor directly based upon the error values for the pixels at each of the respective distances from the surface and the ground truth measurements.

18. The method of claim 17 , further comprising:

prior to averaging the depth values of the pixels over the depth images in each of the subsets captured at the respective distances from the surface, for each of the depth images of the surface captured by the depth sensor:

detecting vertical bands in a depth image;

determining respective shift values in the depth values for the vertical bands in the depth image; and

removing the shift values from the corresponding depth values in the depth image.

19. The method of claim 17 , further comprising:

computing averaged error values for patches of the pixels at each of the respective distances from the surface;

wherein the patches comprise respective pluralities of the pixels; and

wherein respective subsets of the depth calibration data are generated for the patches based upon the averaged error values for the patches of the pixels at each of the respective distances from the surface and the ground truth measurements.

20. The method of claim 17 , wherein the depth calibration data for the non-linear error model calibrated for the depth sensor comprises a parameter matrix, and wherein the parameter matrix comprises error fitting parameters for the pixels.