Apparatus and method for fusing images
View Patent ↗Provided is a method of merging or fusing a visible image captured by a visible camera and an infrared (IR) image captured by an IR camera. The method includes: providing a first image of a subject captured by a first camera and a second image of the subject captured by a second camera; converting brightness of the second image based on brightness of the first image; and merging the first image and the second image having the converted brightness.
1. A method of merging images, the method comprising:
providing a first image of a subject captured by a first camera and a second image of the subject captured by a second camera;
calculating a brightness average of the first image;
converting brightness of the second image based on comparison between the brightness average of the first image and the brightness of the second image; and
merging the first image and the second image having the converted brightness,
wherein the converting the brightness of the second image comprises:
performing brightness attenuation processing according to the following equation:
x IRt =[( x IR −m VISIBLE )/ m VISIBLE +1] 2 , if x IR ≦m VISIBLE ; or
performing brightness enhancement processing according to the following equation:
x IRt =[( x IR −m VISIBLE )× m VISIBLE +1] 2 , if x IR >m VISIBLE
where x IRt denotes brightness of the converted second image, x IR denotes the brightness of the second image and m VISIBLE denotes the brightness average of the first image.
2. The method of claim 1 , wherein the providing the first and second image comprises:
matching the first image;
matching the second image captured at a different wavelength range from the first image; and
providing the matched first and second images.
3. The method of claim 1 , wherein the first camera comprises a visible camera, and the second camera comprises an infrared (IR) camera.
4. The method of claim 1 , further comprising calculating a brightness standard deviation of each of the first and second images.
5. The method of claim 1 , wherein the performing the brightness enhancement processing comprises multiplying a difference between a-brightness of the second image and the brightness average of the first image by the brightness average of the first image.
6. The method of claim 1 , wherein the performing the brightness attenuation processing comprises dividing a difference between brightness of the second image and the brightness average of the first image by the brightness average of the first image.
7. The method of claim 1 , further comprising:
calculating a first weight to apply the first weight to the first image when merging the first and second images in response to brightness of the first image being higher than a first threshold value; and
calculating a second weight to apply the second weight to the second image when merging the first and second images in response to the converted brightness of the second image being higher than a second threshold value.
8. The method of claim 7 , wherein the first threshold value is calculated by adding a brightness average of the second image to a product of a first constant and a brightness standard deviation of the second image, and
wherein the second threshold value is calculated by adding a brightness average of the first image to a product of a second constant and a brightness standard deviation of the first image.
9. An apparatus for merging images, the apparatus comprising:
a first camera configured to capture a first image of a subject;
a second camera configured to capture a second image of the subject, the second image captured at a different wavelength range from the first camera;
a statistical information calculator configured to calculate a brightness average of the first image;
a converter configured to convert brightness of the second image based on comparison between the brightness average of the first image and brightness of the second image; and
a merger configured to merge the first image and the second image having the converted brightness,
wherein the converter is configured to perform brightness attenuation processing according to the following equation:
x IRt =[( x IR −m VISIBLE )/ m VISIBLE +1] 2 , if x IR ≦m VISIBLE or
configured to perform the brightness enhancement processing according to the following equation:
x IRt =[( x IR −m VISIBLE )× m VISIBLE +1] 2 , if x IR >m VISIBLE ,
where x IRt denotes brightness of the converted second image, x IR denotes the brightness of the second image and m VISIBLE denotes the brightness average of the first image.
10. The apparatus of claim 9 further comprising a matching unit configured to match the first image of the subject and configured to match the second image of the subject.
11. The apparatus of claim 9 , wherein the first camera comprises a visible camera, and the second camera comprises an IR camera.
12. The apparatus of claim 10 , wherein the statistical information calculator is configured to calculate a brightness standard deviation of each of the matched first and second images.
13. The apparatus of claim 9 , wherein the converter is configured to perform the brightness enhancement processing by multiplying a difference between brightness of the second image and the brightness average of the first image by the brightness average of the first image.
14. The apparatus of claim 9 , wherein the converter is configured to perform the brightness attenuation processing by dividing a difference between a brightness of the second image and the brightness average of the first image by the brightness average of the first image.
15. The apparatus of claim 9 , further comprising:
a weight calculator configured to calculate a first weight to apply the first weight to the first image when the merger merges the first and second images in response to brightness of the first image being higher than a first threshold value and configured to calculate a second weight to apply the second weight to the second image when the merger merges the first and second images in response to the converted brightness of the second image being higher than a second threshold value.
16. The apparatus of claim 15 , wherein the first threshold value is calculated by adding a brightness average of the second image to a product of a first constant and a brightness standard deviation of the second image, and
wherein the second threshold value is calculated by adding a brightness average of the first image to a product of a second constant and a brightness standard deviation of the first image.
17. A non-transitory computer-readable recording medium storing a computer program for executing the method of claim 1 .