Image processing device and image processing method
Provided is an image processing device that acquires a stereo image captured by a plurality of cameras each including a wide-angle lens, divides viewing angles of the cameras with reference to optical axes corrected to be parallel to each other, generates a plurality of base images in each of which a range of each divided viewing angle is reflected and a plurality of reference images on a basis of wide-angle images constituting the stereo image, applies projective transformation to the reference images, and calculates a distance to a predetermined object on a basis of corresponding image pairs of the plurality of base images and the plurality of reference images after projective transformation.
1. An image processing device, comprising:
an acquisition unit configured to acquire a stereo image captured by a plurality of cameras, wherein
each camera of the plurality of cameras includes a wide-angle lens, and
the stereo image includes a first wide-angle image captured by a first camera of the plurality of cameras and a second wide-angle image captured by a second camera of the plurality of cameras;
a projection unit configured to project the first wide-angle image onto a first virtual spherical surface and the second wide-angle image onto a second virtual spherical surface;
a correction unit configured to correct, based on the stereo image, an optical axis of the first camera and an optical axis of the second camera,
wherein the corrected optical axis of the first camera is parallel to the corrected optical axis of the second camera;
a generation unit configured to:
divide viewing angles of the plurality of cameras with reference to the corrected optical axis of the first camera and the corrected optical axis of the second camera;
reproject the projected first wide-angle image onto a first plurality of planes that is on the first virtual spherical surface;
reproject the projected second wide-angle image onto a second plurality of planes that is on the second virtual spherical surface;
generate a plurality of base images based on the reprojection of the projected first wide-angle image onto the first plurality of planes; and
generate a plurality of reference images based on the reprojection of the projected second wide-angle image onto the second plurality of planes,
wherein each base image of the plurality of base images reflects a range of a corresponding viewing angle of the divided viewing angles;
a projective transformation unit configured to apply projective transformation to each reference image of the plurality of reference images; and
a distance calculation unit configured to calculate, after the projective transformation, a distance to a first object based on corresponding image pairs of the plurality of base images and the plurality of reference images.
2. The image processing device according to claim 1 , wherein the first wide-angle image and the second wide-angle image include the viewing angles of the plurality of cameras.
3. The image processing device according to claim 1 , further comprising a storage unit configured to store information regarding the corrected optical axis of the first camera and the corrected optical axis of the second camera, wherein the stereo image includes a second object.
4. The image processing device according to claim 3 , further comprising a parameter generation unit configured to generate a parameter for the projective transformation, based on corresponding points of a base image of the plurality of base images and a corresponding reference image of the plurality of reference images, wherein
the base image and the corresponding reference image constitute an image pair of the corresponding image pairs, and
the storage unit is further configured to store the parameter for the projective transformation.
5. The image processing device according to claim 4 , wherein
the correction unit is further configured to set the corrected optical axis of the first camera and the corrected optical axis of the second camera, based on the stored information regarding the corrected optical axis of the first camera and the corrected optical axis of the second camera, and
the projective transformation unit is further configured to apply the projective transformation to each reference image of the plurality of reference images based on the stored parameter.
6. The image processing device according to claim 3 , wherein the correction unit is further configured to repeatedly correct the optical axis of the first camera and the optical axis of the second camera until a correction error becomes equal to or less than a threshold value.
7. The image processing device according to claim 1 , further comprising the plurality of cameras.
8. An image processing method, comprising:
acquiring a stereo image captured by a plurality of cameras, wherein
each camera of the plurality of cameras includes a wide-angle lens, and
the stereo image includes a first wide-angle image captured by a first camera of the plurality of cameras and a second wide-angle image captured by a second camera of the plurality of cameras;
projecting the first wide-angle image onto a first virtual spherical surface and the second wide-angle image onto a second virtual spherical surface;
correcting, based on the stereo image, an optical axis of the first camera and an optical axis of the second camera,
wherein the corrected optical axis of the first camera is parallel to the corrected optical axis of the second camera;
dividing viewing angles of the plurality of cameras with reference to the corrected optical axis of the first camera and the corrected optical axis of the second camera;
reprojecting the projected first wide-angle image onto a first plurality of planes that is on the first virtual spherical surface;
reprojecting the projected second wide-angle image onto a second plurality of planes that is on the second virtual spherical surface;
generating a plurality of base images based on the reprojection of the projected first wide-angle image onto the first plurality of planes; and
generating a plurality of reference images based on the reprojection of the projected second wide-angle image onto the second plurality of planes,
wherein each base image of the plurality of base images reflects a range of a corresponding viewing angle of the divided viewing angles;
applying projective transformation to each reference image of the plurality of reference images; and
calculating, after the projective transformation, a distance to a specific object based on corresponding image pairs of the plurality of base images and the plurality of reference images.
9. A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor, cause the processor to execute operations, the operations comprising:
acquiring a stereo image captured by a plurality of cameras, wherein
each camera of the plurality of cameras includes a wide-angle lens, and
the stereo image includes a first wide-angle image captured by a first camera of the plurality of cameras and a second wide-angle image captured by a second camera of the plurality of cameras;
projecting the first wide-angle image onto a first virtual spherical surface and the second wide-angle image onto a second virtual spherical surface;
correcting, based on the stereo image, an optical axis of the first camera and an optical axis of the second camera,
wherein the corrected optical axis of the first camera is parallel to the corrected optical axis of the second camera;
dividing viewing angles of the plurality of cameras with reference to the corrected optical axis of the first camera and the corrected optical axis of the second camera;
reprojecting the projected first wide-angle image onto a first plurality of planes that is on the first virtual spherical surface;
reprojecting the projected second wide-angle image onto a second plurality of planes that is on the second virtual spherical surface;
generating a plurality of base images based on the reprojection of the projected first wide-angle image onto the first plurality of planes; and
generating a plurality of reference images based on the reprojection of the projected second wide-angle image onto the second plurality of planes,
wherein each base image of the plurality of base images reflects a range of a corresponding viewing angle of the divided viewing angles;
applying projective transformation to each reference image of the plurality of reference images; and
calculating, after the projective transformation, a distance to a specific object based on corresponding image pairs of the plurality of base images and the plurality of reference images.