Microlens amplitude masks for flying pixel removal in time-of-flight imaging
The microlens amplitude masks for flying pixel removal in time-of-flight imaging includes systems, devices, methods, and instructions for image depth determination, including receiving an image, adding noise to the image, determining a set of correlation images, each correlation image having a varying phase offset, for each pixel of the image, generating a masked pixel by applying a mask array, and for each masked pixel, determining the depth of the masked pixel to generate a depth map for the image on a per pixel basis.
1. A computer-implemented method for image depth determination, the computer-implemented method comprising:
receiving an image;
adding noise to the image;
for the image, determining a set of correlation images, each correlation image having a varying phase offset
for each pixel of the image, generating a masked pixel by applying a mask array; and
for each masked pixel, determining the depth of the masked pixel to generate a depth map for the image on a per pixel basis, wherein neighboring pixels have different apertures, wherein a first neighboring pixel has a different signal-to-noise ratio than a second neighboring pixel.
2. The computer-implemented method according to claim 1 , wherein the image is captured at a sensor having a mask pattern.
3. The computer-implemented method according to claim 2 , wherein the mask pattern is photolithographically applied to each pixel of the sensor.
4. The computer-implemented method according to claim 1 , wherein each pixel of the image is captured using a respective aperture.
5. The computer-implemented method according to claim 1 , wherein the first neighboring pixel has a different susceptibility to being a flying pixel than the second neighboring pixel.
6. The computer-implemented method according to claim 1 , wherein the noise is added according to a Poisson distribution or a Skellam distribution.
7. The computer-implemented method according to claim 1 , further comprising:
processing each masked pixel by a convolution refinement network.
8. The computer-implemented method according to claim 7 , wherein the convolution refinement network receives a first depth estimate for each pixel and outputs the depth for each pixel.
9. The computer-implemented method according to claim 7 , wherein the convolution refinement network outputs a correction to the mask array based upon the difference between the first depth estimate for each pixel and the depth for each pixel.
10. A time-of-flight system for image depth determination, the system comprising:
a lens configured to receive incident light and
a light sensor having a plurality of pixels, the light sensor configured to receive the incident light through a plurality of masks, each pixel corresponding to a respective mask that selectively blocks incident light paths to provide a differentiable apertures for neighboring pixels,
wherein a first neighboring pixel has a different signal-to-noise ratio than a second neighboring pixel.
11. The system according to claim 10 , wherein the first neighboring pixel has a different susceptibility to being a flying pixel than the second neighboring pixel.