Hierarchical disparity hypothesis generation with slanted support windows

1. A method, comprising:

capturing a first image and a second image of a scene using at least one imaging camera of an imaging system, wherein the first image and the second image form a stereo image pair and each comprises a plurality of pixels;

initializing, at a processor of the imaging system, each of the plurality of pixels in the second image with a disparity hypothesis;

recursively determining, at the processor, from an image tile of a smaller subset of pixels of the second image to an image tile of a larger subset of pixels of the second image, matching costs of the disparity hypothesis for each of the plurality of pixels in the second image to generate an initial tiled disparity map including a plurality of image tiles, wherein each image tile of the initial tiled disparity map is assigned a disparity value estimate;

refining, at the processor, the disparity value estimate of each image tile and further associating the disparity value estimate of each image tile with a slant estimate;

determining, at the processor, a final disparity estimate for each pixel of the second image based on the refined disparity value estimate of each image tile; and

generating a depth map for the scene using correspondence between the plurality of pixels of the first image and the second image and based on the determined final disparity estimate for each pixel of the second image.

2. The method of claim 1 , wherein initializing each of the plurality of pixels in the second image with a disparity hypothesis comprises:

seeding each of the plurality of pixels in the second image with a set of disparity hypotheses; and

comparing, for a first pixel of the plurality of pixels, matching costs between hypotheses in the set of disparity hypotheses relative to a disparity between a corresponding pixel in the stereo image pair; and

storing one of the set of disparity hypotheses with a lowest matching cost as the disparity hypothesis.

3. The method of claim 2 , wherein recursively determining matching costs of the disparity hypothesis for each of the plurality of pixels in the second image further comprises:

computing, for the image tile of the smaller subset of pixels, matching costs associated with the disparity hypothesis for pixels associated with the image tile of the smaller subset of pixels; and

storing, for the image tile of the smaller subset of pixels, a hypothesis identified as having the lowest matching cost; and

providing the identified hypothesis to a next level of recursion.

4. The method of claim 3 , further comprising:

computing, for the image tile of the larger subset of pixels, matching costs associated with the disparity hypothesis for pixels associated with the image tile of the larger subset of pixels, wherein the disparity hypothesis includes the identified hypothesis from the image tile of the smaller subset of pixels;

storing, for the image tile of the larger subset of pixels, the identified hypothesis having the lowest matching cost; and

providing the identified hypothesis to the next level of recursion.

5. The method of claim 1 , wherein refining the disparity value estimate of each image tile further includes replacing the disparity value estimate and the slant estimate for at least one image tile with the disparity value estimate and the slant estimate of a neighboring image tile having a lower reconstruction cost.

6. The method of claim 1 , wherein determining the final disparity estimate for each pixel of the second image further comprises:

expanding each image tile of the initial tiled disparity map to overlap a plurality of neighboring image tiles; and

generating, based on the expanded image tiles, a plurality of integral tiles.

7. The method of claim 6 , further comprising:

performing, for each one pixel of the expanded image tiles, a plurality of parabola fits of cost volumes using the plurality of integral tiles;

computing a reconstruction error over patches centered over each one pixel of the expanded image tiles; and

saving a smallest reconstruction error as a final disparity estimate for each pixel of the initial tiled disparity map.

8. An electronic device, comprising:

a pair of cameras to capture a first image and a second image of a scene, wherein the first image and the second image form a stereo image pair and each comprises a plurality of pixels, and further wherein each of the plurality of pixels in the first image is associated with a disparity value; and

a processor configured to:

initialize each of the plurality of pixels in the second image with a disparity hypothesis;

recursively determine, from an image tile of a smaller subset of pixels of the second image to an image tile of a larger subset of pixels of the second image, matching costs of the disparity hypothesis for each of the plurality of pixels in the second image to generate an initial tiled disparity map including a plurality of image tiles, wherein each image tile of the initial tiled disparity map is assigned a disparity value estimate;

refine the disparity value estimate of each image tile and associate the disparity value estimate of each image tile with a slant estimate;

determine, based on the refined disparity value estimate of each image tile, a final disparity estimate for each pixel of the second image; and

generate a depth map for the scene, using correspondence between the plurality of pixels of the first image and the second image and based on the determined final disparity estimate for each pixel of the second image.

9. The electronic device of claim 8 , wherein the processor is further configured to:

seed each of the plurality of pixels in the second image with a set of disparity hypotheses;

compare, for a first pixel of the plurality of pixels, matching costs between hypotheses in the set of disparity hypotheses relative to a disparity between a corresponding pixel in the stereo image pair; and

store one of the set of disparity hypotheses with a lowest matching cost as the disparity hypothesis.

10. The electronic device of claim 9 , wherein the processor is further configured to:

compute, for the image tile of the smaller subset of pixels, matching costs associated with the disparity hypothesis for pixels associated with the image tile of the smaller subset of pixels; and

store, for the image tile of a smaller subset of pixels, a hypothesis identified as having the lowest matching cost; and

provide the identified hypothesis to a next level of recursion.

11. The electronic device of claim 10 , wherein the processor is further configured to:

compute, for the image tile of the larger subset of pixels, matching costs associated with the disparity hypothesis for pixels associated with the image tile of the larger subset of pixels, wherein the disparity hypothesis includes the identified hypothesis from the image tile of the smaller subset of pixels;

store, for the image tile of the larger subset of pixels, the identified hypothesis having the lowest matching cost; and

provide the identified hypothesis to the next level of recursion.

12. The electronic device of claim 8 , wherein the processor is further configured to:

replace the disparity value estimate and the slant estimate for at least one image tile with the disparity value estimate and the slant estimate of a neighboring image tile having a lower reconstruction cost.

13. The electronic device of claim 8 , wherein the processor is further configured to:

expand each image tile of the initial tiled disparity map to overlap a plurality of neighboring image tiles; and

generate, based on the expanded image tiles, a plurality of integral tiles.

14. The electronic device of claim 13 , wherein the processor is further configured to:

perform, for each one pixel of the expanded image tiles, a plurality of parabola fits of cost volumes using the plurality of integral tiles;

compute a reconstruction error over patches centered over each one pixel of the expanded image tiles; and

save a smallest reconstruction error as a final disparity estimate for each pixel of the initial tiled disparity map.

15. A system, comprising:

a pair of cameras to capture a first image and a second image of a scene, wherein the first image and the second image form a stereo image pair and each comprises a plurality of pixels, and further wherein each of the plurality of pixels in the first image is associated with a disparity value;

an initialization module configured to initialize each of the plurality of pixels in the second image with a disparity hypothesis, wherein the initialization module is further configured to recursively determine, from an image tile of a smaller subset of pixels of the second image to an image tile of a larger subset of pixels of the second image, matching costs of the disparity hypothesis for each of the plurality of pixels in the second image to generate an initial tiled disparity map including a plurality of image tiles, wherein each image tile of the initial tiled disparity map is assigned a disparity value estimate;

a propagation module configured to refine the disparity value estimate of each image tile and to associate the disparity value estimate of each image tile with a slant estimate; and

a refinement module configured to determine, based on the refined disparity value estimate of each image tile, a final disparity estimate for each pixel of the second image.

16. The system of claim 15 , wherein the initialization module is further configured to:

seed each of the plurality of pixels in the second image with a set of disparity hypotheses;

compare, for a first pixel of the plurality of pixels, matching costs between hypotheses in the set of disparity hypotheses relative to a disparity between a corresponding pixel in the stereo image pair; and

store one of the set of disparity hypotheses with a lowest matching cost as the disparity hypothesis.

17. The system of claim 16 , wherein the initialization module is further configured to:

compute, for the image tile of the smaller subset of pixels, matching costs associated with the disparity hypothesis for pixels associated with the image tile of the smaller subset of pixels; and

store, for the image tile of a smaller subset of pixels, a hypothesis identified as having the lowest matching cost; and

provide the identified hypothesis to a next level of recursion.

18. The system of claim 17 , wherein the initialization module is further configured to:

compute, for the image tile of the larger subset of pixels, matching costs associated with the disparity hypothesis for pixels associated with the image tile of the larger subset of pixels, wherein the disparity hypothesis includes the identified hypothesis from the image tile of the smaller subset of pixels;

store, for the image tile of the larger subset of pixels, the identified hypothesis having the lowest matching cost; and

provide the identified hypothesis to the next level of recursion.

19. The system of claim 15 , wherein the propagation module is further configured to:

replace the disparity value estimate and the slant estimate for at least one image tile with the disparity value estimate and the slant estimate of a neighboring image tile having a lower reconstruction cost.

20. The system of claim 15 , wherein the refinement module is further configured to:

expand each image tile of the initial tiled disparity map to overlap a plurality of neighboring image tiles;

generate, based on the expanded image tiles, a plurality of integral tiles;

perform, for each one pixel of the expanded image tiles, a plurality of parabola fits of cost volumes using the plurality of integral tiles;

compute a reconstruction error over patches centered over each one pixel of the expanded image tiles; and

save a smallest reconstruction error as a final disparity estimate for each pixel of the initial tiled disparity map.