Image processing device, imaging device, and image processing method

1. An image processing device which creates a depth map using a multi-view image, the image processing device comprising:

a detection unit configured to detect one or more features in each of images included in the multi-view image, by performing edge detection or corner detection on the multi-view image;

a derivation unit configured to derive, as corresponding points, a pair of the features corresponding to each other between the images included in the multi-view image, based on similarity in feature quantity of the detected features;

a calculation unit configured to calculate, based on the derived corresponding points, a reliability of a correspondence between the images included in the multi-view image in preparation for creating the depth map;

a creation unit configured to create the depth map using the correspondence between the images included in the multi-view image when the calculated reliability is a first reliability, and create the depth map without using the correspondence between the images included in the multi-view image when the calculated reliability is a second reliability which is lower than the first reliability; and

a transformation unit configured to calculate, using at least four of the derived corresponding points, a transformation parameter for projective transformation of a first image included in the multi-view image into a second image included in the multi-view image, and transform the first image using the transformation parameter, wherein:

the detection unit is configured to detect an edge as the feature, and

the calculation unit is configured to calculate the reliability which is higher when an error of the first image resulting from the projective transformation with respect to the second image is smaller, and calculate the error of the first image resulting from the projective transformation with respect to the second image, by calculating a difference in pixel values only from a region where the detected edge has a strength above a threshold value in the second image.

2. The image processing device according to claim 1 ,

wherein the transformation unit is configured to calculate plural transformation parameters using plural combinations of M corresponding points selected from among N of the derived corresponding points, and transform the first image into plural first images resulting from the projective transformation using the plural transformation parameters, where N≧5 and 4≦M≦N, the combinations being different from each other, and

wherein the calculation unit is configured to calculate, as the error, one of a root mean square, an average absolute value, a square sum, and a sum of absolute values of differences between the plural transformed first images and the second image, and calculate the reliability which is higher when the error is smaller.

3. The image processing device according to claim 1 ,

wherein when the calculated reliability is the first reliability, the creation unit is configured to create the depth map by calculating disparity between the images included in the multi-view image, based on similarity between local regions.

4. The image processing device according to claim 1 ,

wherein when the calculated reliability is the second reliability, the creation unit is configured to create the depth map by estimating depth using at least one of geometric information, color information, and information on a motion of an object in one of the images included in the multi-view image.

5. The image processing device according to claim 1 ,

wherein the multi-view image is included in a multi-view video, and

when the calculated reliability is the second reliability, the creation unit is configured to create the depth map for the multi-view image using a depth map created using an other multi-view image which is included in the multi-view video and temporally different from the multi-view image.

6. The image processing device according to claim 5 ,

wherein when the calculated reliability is the second reliability, the creation unit is configured to create the depth map using both a depth map created using a correspondence between local regions in the other multi-view image and depth estimated using at least one of geometric information, color information, and information on a motion of an object in one of the images included in the multi-view image.

7. The image processing device according to claim 5 ,

wherein when the calculated reliability is the second reliability, the creation unit is configured to use, as the depth map for the multi-view image, a depth map created using a correspondence between local regions in the other multi-view image.

8. The image processing device according to claim 1 ,

wherein the multi-view image is a stereogram,

wherein the transformation unit is further configured to rectify the stereogram to parallelize epipolar lines in the stereogram, and

when the calculated reliability is the first reliability, the calculation unit is configured to create the depth map using a correspondence in the rectified stereogram.

9. The image processing device according to claim 1 ,

wherein the image processing device is configured as an integrated circuit.

10. An imaging device comprising:

the image processing device according to claim 1 ; and

an imaging unit configured to capture the multi-view image.

11. An image processing method of creating a depth map using a multi-view image, the image processing method comprising:

detecting one or more features in each of images included in the multi-view image, by performing edge detection or corner detection on the multi-view image;

deriving, as corresponding points, a pair of the features corresponding to each other between the images included in the multi-view image, based on similarity in feature quantity of the detected features;

calculating, based on the derived corresponding points, a reliability of a correspondence between the images included in the multi-view image in preparation for creating the depth map;

creating the depth map using the correspondence between the images included in the multi-view image when the calculated reliability is a first reliability, and creating the depth map without using the correspondence between the images included in the multi-view image when the calculated reliability is a second reliability which is lower than the first reliability;

calculating, using at least four of the derived corresponding points, a transformation parameter for projective transformation of a first image included in the multi-view image into a second image included in the multi-view image; and

transforming the first image using the transformation parameter, wherein:

an edge is detected as the feature, and

the reliability which is higher when an error of the first image resulting from the projective transformation with respect to the second image is smaller is calculated, and the error of the first image resulting from the projective transformation with respect to the second image is calculated by calculating a difference in pixel values only from a region where the detected edge has a strength above a threshold value in the second image.

12. A non-transitory computer-readable recording medium for use in a computer, the non-transitory computer-readable recording medium having a computer program recorded thereon for causing the computer to execute the image processing method according to claim 11 .