Image stitching method and apparatus
View Patent ↗The present invention relates to an image processing technology, and discloses an image stitching method and apparatus to solve the problem of severe ghosting of an image stitched in the prior art. In the embodiments of the present invention, the overlap region of two images is found, a depth image of the overlap region is obtained, and the two images are stitched together according to the depth image. In the stitching process, the 3-dimensional information of the images is obtained by using the depth image to deghost the image. The method and apparatus under the present invention are applicable to multi-scene videoconferences and the occasions of making wide-view images or videos.
1. An image stitching method, comprising:
obtaining two images between which an overlap region exists;
searching for the overlap region of the two images;
obtaining a depth image of the overlap region; and
stitching the two images into one image according to the depth image, wherein the stitching of the two images into one image according to the depth image comprising:
calculating a homographic matrix between the two images;
mapping a first image to a target coordinate system according to a mapping relation between the first image and the homographic matrix;
calculating an amount of parallax between corresponding pixels of the two images in the overlap region according to the depth image;
discarding pixel points of a second image if the parallax amount falls within a preset parallax range;
mapping pixel points of the second image in the overlap region to coordinates of corresponding pixel points of the first image in the target coordinate system if the parallax amount exceeds the preset parallax range; and
mapping pixels of the second image outside the overlap region to the target coordinate system according to a mapping relation between the second image and the homographic matrix.
2. The image stitching method according to claim 1 , wherein obtaining the depth image of the overlap region comprises: obtaining the depth image of the overlap region by using a depth camera directly.
3. The image stitching method according to claim 1 , wherein: the target coordinate system is a coordinate system of the first image.
4. The image stitching method according to claim 1 , wherein: the mapping of the pixel points of the second image in the overlap region to the coordinates of the corresponding pixel points of the first image in the target coordinate system comprises: calculating a weighted average chrominance value and a weighted average luminance value of the corresponding pixel points in the two images, and using the weighted average chrominance value and the weighted average luminance value as a chrominance value and a luminance value of the corresponding pixel points of the first image in the target coordinate system respectively.
5. An image stitching apparatus, comprising:
a first obtaining unit configured to obtain two images between which an overlap region exists;
a searching unit configured to search for the overlap region of the two images;
a second obtaining unit configured to obtain a depth image of the overlap region; and
a stitching unit configured to stitch the two images into one image according to the depth image; wherein the stitching unit comprises:
a first calculating module configured to calculate a homographic matrix between the two images;
a second calculating module configured to calculate a parallax amount between corresponding pixels of the two images in the overlap region according to the depth image; and
a mapping module configured to:
map a first image to a target coordinate system according to a mapping relation between the first image and the homographic matrix;
map pixel points of a second image in the overlap region to coordinates of corresponding pixel points of the first image in the target coordinate system, if the parallax amount exceeds a preset parallax range; and
map pixels of the second image outside the overlap region to the target coordinate system according to a mapping relation between the second image and the homographic matrix.
6. The image stitching apparatus according to claim 5 , wherein: the second obtaining unit is a depth camera.
7. The image stitching apparatus according to claim 5 , wherein the second obtaining unit comprises: a calculating module configured to calculate a parallax amount between corresponding pixels in the overlap region of the two images, and calculate the depth image of the overlap region according to the parallax amount.
8. The image stitching apparatus according to claim 5 , wherein the second obtaining unit comprises: an obtaining module configured to obtain two new images in the overlap region at two viewpoints; and a calculating module configured to calculate a parallax amount between corresponding pixels in the two new images, and calculate the depth image in the overlap region according to the parallax amount.
9. The image stitching apparatus according to claim 5 , wherein:
the mapping module is configured to:
calculate a weighted average chrominance value and a weighted average luminance value of the corresponding pixel points in the two images, if the parallax amount exceeds the preset parallax range; and
use the weighted average chrominance value and the weighted average luminance value as a chrominance value and a luminance value of the corresponding pixel points of the first image in the target coordinate system, respectively.
10. The image stitching method according to claim 1 , wherein obtaining the depth image of the overlap region comprises:
calculating a parallax amount between corresponding pixels in the overlap region of the two images; and
calculating the depth image of the overlap region according to the parallax amount.
11. The image stitching method according to claim 1 , wherein obtaining the depth image of the overlap region comprises: obtaining two new images in the overlap region at two viewpoints; calculating a parallax amount between corresponding pixels in the two new images; and calculating the depth image in the overlap region according to the parallax amount.