Method and apparatus for fusion of images
A method and an apparatus for improving a main image by fusing the richer information contained in a secondary image are described. A 3D structure of objects contained in the secondary image is retrieved and a parallax-corrected version of the secondary image is generated using the 3D structure. For this purpose a camera pose for which a projection of the 3D structure of the objects contained in the secondary image best resembles the perspective in the main image is determined and the parallax-corrected version of the secondary image is synthesized based on the determined camera pose. The parallax-corrected version of the secondary image is then fused with the main image.
1. A method for fusing a secondary image with a main image, the method comprising:
retrieving a 3D structure of objects contained in the secondary image by using information available for the secondary image;
generating a parallax-corrected version of the secondary image using the 3D structure by determining a camera pose for which a projection of the 3D structure of the objects contained in the secondary image resembles the perspective in the main image and synthesizing the parallax-corrected version of the secondary image based on the determined camera pose; and
fusing the parallax-corrected version of the secondary image with the main image.
2. The method according to claim 1 , wherein the 3D structure of the secondary image is retrieved from depth information or a light-field description provided for the secondary image.
3. The method according to claim 1 , wherein the parallax-corrected version of the secondary image is synthesized by performing a photo-realistic warping of the secondary image.
4. The method according to claim 1 , further comprising interpolating a boundary region between the main image and the fused parallax-corrected version of the secondary image.
5. The method according to claim 1 , wherein the main image is a panoramic image and the secondary image is a more detailed image of an object in the panoramic image.
6. The method according to claim 1 , wherein the main image and the secondary image are two images of a set of images.
7. An apparatus for fusing a secondary image with a main image, the apparatus comprising at least one processor configured to:
retrieve a 3D structure of objects contained in the secondary image by using information available for the secondary image;
generate a parallax-corrected version of the secondary image using the 3D structure by determining a camera pose for which a projection of the 3D structure of the objects contained in the secondary image resembles the perspective in the main image and to synthesize the parallax-corrected version of the secondary image based on the determined camera pose; and
fuse the parallax-corrected version of the secondary image with the main image.
8. The apparatus according to claim 7 , wherein the at least one processor is further configured to retrieve the 3D structure of the secondary image from depth information or a light-field description provided for the secondary image.
9. The apparatus according to claim 7 , wherein the at least one processor is further is configured to synthesize the parallax-corrected version of the secondary image by performing a photo-realistic warping of the secondary image.
10. The apparatus according to claim 7 , wherein the at least one processor is further configured to interpolate a boundary region between the main image and the fused parallax-corrected version of the secondary image.
11. The apparatus according to claim 7 , wherein the main image is a panoramic image and the secondary image is a more detailed image of an object in the panoramic image.
12. The apparatus according to claim 7 , wherein the main image and the secondary image are two images of a set of images.