Image fusion system and method

1. An image fusion system comprising:

at least one processor configured to implement:

an image selection unit configured to select at least two images from a plurality of images captured by a plurality of different type cameras;

a first matching unit configured to match the selected images by adjusting, based on zoom information about the selected images, aspect ratios of the selected images to be the same as a reference aspect ratio, wherein the aspect ratios are determined according to field of views (FOVs) to capture the selected images, respectively;

a second matching unit configured to match the selected images by adjusting resolutions of the selected images to be the same as a reference resolution;

a third matching unit configured to match the selected images by applying a mutual geometric relationship between the selected images; and

a fusion unit configured to fuse the selected images which are matched by the first to third matching units, and thus generate a fused image,

wherein the second matching unit is configured to adjust the resolutions of the selected images of which the aspect ratios are adjusted to be the same as the reference aspect ratio, and

wherein the third matching is configured to apply the mutual geometric relationship to the selected images of which the resolutions are adjusted to be the same as the reference resolution.

2. The system of claim 1 , wherein the plurality of different type cameras comprise:

a plurality of pan-tilt-zoom (PTZ) cameras, each capturing the same photography area with respective FOVs; and

a plurality of omnidirectional cameras arrayed at predetermined intervals such that adjacent portions of photography areas captured by two adjacent cameras of the omnidirectional cameras overlap.

3. The system of claim 2 , wherein the plurality of PTZ cameras comprise:

a visible light camera configured to generate a visible image;

a thermal imaging camera configured to generate a thermal image; and

an infrared camera configured to generate an infrared image.

4. The system of claim 2 , wherein the image selection unit selects the at least two images from among a plurality of images captured by the plurality of PTZ cameras and a cropped image generated by cropping an area corresponding to the plurality of images from a panorama image that is generated by using images captured by the plurality of omnidirectional cameras.

5. The system of claim 4 , wherein the zoom information about the selected images includes zoom information about the cropped image which is a digital zoom ratio that is converted from a size of the cropped image.

6. The system of claim 1 , wherein the first matching unit crops left and right sides of the selected images or performs zero padding on the selected images.

7. The system of claim 1 , wherein the third matching unit comprises:

a feature extraction unit configured to detect features of the selected images;

a feature matching unit configured to determine one of the selected images as a reference image, and detect matching features between features of the reference image and at least one image other than the reference image; and

a conversion function estimation unit configured to estimate conversion functions of the at least one image other than the reference image based on the detected matching features.

8. The system of claim 1 , wherein the fusion unit adjusts transparency of the selected images by applying a weight to each of the selected images.

9. The system of claim 1 , wherein the processor is configured to further implement a multiplexer configured to output at least one of the plurality of images and the fused image.

10. An image fusion system comprising:

a plurality of cameras configured to generate a plurality of images, respectively; and

at least one processor configured to implement an image fusion unit configured to fuse the plurality of images into a single image,

wherein fields of views (FOVs) of the plurality of cameras are set to be the substantially same,

wherein the plurality of cameras include at least two cameras among a visible light camera, a thermal imaging camera and an infrared light camera, and

wherein the image fusion unit comprises:

a resizing unit configured to control the plurality of images such that respective sizes of the plurality of images are the same; and

an information fusion unit configured to extract an object that is not displayed in a visible image captured by the visible light camera but displayed in a thermal image captured by the thermal imaging camera and an object that is not displayed in the visible and thermal images but displayed in an infrared image captured by the infrared light camera, and overlap the thermal image and the infrared image on the visible image so that extracted objects are displayed on the visible image.

11. An image fusion method comprising:

selecting at least two images from a plurality of images captured by a plurality of different type cameras; first matching the selected images by adjusting, based on zoom information about selected images, aspect ratios of the selected images to be the same as a reference aspect ratio, wherein the aspect ratios are determined according to field of views (FOVs) to capture the selected images, respectively;

second matching the selected images by adjusting resolutions of the selected images to be the same as a reference resolution;

third matching the selected images by applying a mutual geometric relationship between the selected images; and

fusing the selected images which are matched by the three matchings to generate a fused image,

wherein the second matching is performed by adjusting the resolutions of the selected images of which the aspect ratios are adjusted to be the same as the reference aspect ratio, and

wherein the third matching is performed by applying the mutual geometric relationship to the selected images of which the resolutions are adjusted to be the same as the reference resolution.

12. The method of claim 11 , wherein the plurality of different type cameras comprises:

a plurality of pan-tilt-zoom (PTZ) cameras, each capturing the same photography area with respective FOVs; and

a plurality of omnidirectional cameras arrayed at predetermined intervals such that adjacent portions of photography areas captured by two adjacent cameras of the omnidirectional cameras overlap, and

wherein the plurality of PTZ cameras comprise a visible light camera generating a visible image, a thermal imaging camera generating a thermal image, and an infrared light camera generating an infrared image.

13. The method of claim 12 , wherein the selecting the at least two images comprises selecting at least two images from among a plurality of images captured by the plurality of PTZ cameras and a cropped image generated by cropping an area corresponding to the plurality of images from a panorama image that is generated by using images captured by the plurality of omnidirectional cameras.

14. The method of claim 13 , wherein the zoom information about the selected images includes zoom information about the cropped image which is a digital zoom ratio that is converted from a size of the cropped image.

15. The method of claim 11 , wherein the third matching comprises:

detecting features of the selected images;

determining one of the selected images as a reference image and detecting matching features between features of the reference image and at least one image other than the reference image; and

estimating conversion functions of the at least one image other than the reference image based on the detected matching features.

16. The method of claim 11 , wherein the fusing comprises adjusting transparency of the selected images by applying a weight to each of the selected images.