Systems and methods for providing depth map information
A method for providing depth map information based on image data descriptive of a scene. In one embodiment, after generating an initial sequence of disparity map data, performing a smoothing operation or an interpolation to remove artifact introduced in the disparity map data as a result of segmenting the image data into superpixels.
1. A method for providing depth map information based on image data of a scene, the method comprising:
performing pixel-wise local matching among pixels in the image data to determine correspondence based at least in part on lowest matching cost determinations;
segmenting the image data into superpixels, each superpixel comprising a plurality of the pixels;
assigning matching costs to the superpixels based at least in part on the pixel-wise local matching cost determinations;
aggregating the matching costs of the superpixels;
performing a matching cost optimization by iteratively refining disparities between superpixels to generate initial disparity map data representative of an initial disparity map; and
after generating the initial disparity map data representative of the initial disparity map, performing at least one of:
a smoothing operation comprising overwriting at least a portion of the initial disparity map data based at least in part on a normalized convolution to provide modified disparity data representative of a modified disparity map, the modified disparity map comprising an artifact that is smoothed relative to presence of the artifact in the initial disparity map, or
an interpolation based at least in part on the initial disparity map data and the modified disparity map data to reduce presence of an artifact in the modified disparity map data relative to presence of the artifact in the initial disparity map data.
2. The method of claim 1 , further comprising performing both the smoothing operation and the interpolation.
3. The method of claim 2 , wherein the interpolation is performed after the smoothing operation.
4. The method of claim 2 , wherein the interpolation is performed before the smoothing operation.
5. The method of claim 2 , wherein the disparity map data comprises data in a first domain, and further comprising performing a data domain transfer before the interpolation is performed.
6. The method of claim 2 , wherein the smoothing operation is performed on a convolution approximating a Gaussian blur operation by repeatedly applying a box blur operation to the data.
7. The method of claim 6 , wherein the disparity map data comprises data in a first domain and further comprising performing a data domain transfer before the smoothing operation is performed.
8. The method of claim 1 , wherein after generating the initial disparity map data, the interpolation is performed.
9. The method of claim 8 , wherein the interpolation comprises providing a revised sequence ƒ(t) of disparity map data values for the superpixels by replacing missing or corrupt superpixel data values in the initial disparity map data with values of zero.
10. The method of claim 9 , wherein performing the interpolation further comprises convolving a certainty map c(t) for the revised sequence ƒ(t) with a smoothing function g(t) as a normalizing factor to create a final disparity map based at least in part on the modified disparity map data.
11. The method of claim 10 where said convolving is in accord with
f
(
t
)
o
=
f
(
t
)
×
g
(
t
)
c
(
t
)
×
g
(
t
)
for the final disparity map characterized by the reduced artifact relative to a level of artifact present in the initial disparity map data.
12. The method of claim 10 , wherein the final disparity map comprises a reduced artifact relative to an artifact present in the initial disparity map.
13. The method of claim 10 , wherein an element of the certainty map c(t) comprises a value of zero for the missing or corrupt data values and comprises a value of one for data values not missing or corrupt.
14. The method of claim 1 , wherein performing the matching cost optimization comprises defining the superpixels based at least in part on a segmentation size criterion, the segmentation size criterion defining the superpixels according to one or more of: cross intensity boundaries or object boundaries within the scene.
15. The method of claim 14 , wherein defining the superpixels based at least in part on the segmentation size criterion causes creation of an artifact in the initial disparity map data.
16. A method of generating a depth map, the method comprising:
receiving image data comprising a first superpixel and a second superpixel;
performing a matching cost optimization by iteratively refining a difference between the first superpixel and the second superpixel in the image data;
generating, based at least in part on the matching cost optimization, first disparity map data representing a first disparity map;
generating mutual structure data based at least in part on the first disparity map data and the image data;
filtering the mutual structure data to modify a pixel value for an occluded region or a depth discontinuity in the first disparity map; and
generating, based at least in part on filtering the mutual structure data, second disparity map data representative of a second disparity map comprising the modified pixel value for the occluded region or the depth discontinuity.
17. The method of claim 16 , wherein the modified pixel value corrects a depth discontinuity or restores an edge or corner region in the first disparity map.
18. A method of generating a depth map, the method comprising:
receiving image data comprising a first superpixel and a second superpixel;
performing a matching cost optimization by iteratively refining a difference between the first superpixel and the second superpixel in the image data;
generating, based at least in part on the matching cost optimization, first disparity map data representing a first disparity map;
generating mutual structure data based at least in part on the first disparity map data and the image data;
overwriting the mutual structure data to correct an artifact in the first disparity map; and
generating second disparity map data representative of a second disparity map comprising the corrected artifact.
19. The method of claim 18 , wherein the first disparity map is associated with a first domain and the second disparity map is associated with a second domain different from the first domain.