Image fusing method
View Patent ↗Provided is an image fusing apparatus for fusing a thermal image and a visible image of a subject. The image fusing apparatus includes: a determination processor configured to determine importance of each pixel, based on a luminance value thereof, of the thermal image and the visible image; a pixel coefficient setting processor configured to set a pixel coefficient for each pixel of the thermal image based on the importance of each pixel of the thermal image and the visible image; a thermal image processor configured to generate another thermal image by applying the pixel coefficient to the luminance value of each pixel of the thermal image; and an image fusing processor configured to fuse the other thermal image and the visible image to generate a fused image.
1. An image fusing method for fusing a thermal image and a visible image of a subject, the image fusing method comprising:
detecting an important pixel of the thermal image based on a luminance value of each pixel of the thermal image;
detecting a first important pixel and a second important pixel of the visible image based on a luminance value of each pixel of the visible image;
setting a pixel coefficient for each pixel of the thermal image according to whether each pixel of the thermal image is the important pixel and whether each pixel of the visible image is at least one of the first important pixel and the second important pixel;
generating another thermal image by applying the pixel coefficient to the luminance value of each pixel of the thermal image; and
fusing the other thermal image and the visible image to generate a fused image, at an image fusing controller,
wherein the detecting the important pixel comprises:
determining the important pixel of the thermal image of which a luminance value is equal to or greater than a first reference value calculated based on a mean of luminance values of the thermal image;
determining the first important pixel of the visible image of which a luminance value is equal to or greater than a second reference value calculated based on a mean of luminance values of the visible image; and
determining the second important pixel of the visible image of which a luminance value is equal to or greater than a third reference value calculated based on a mean of luminance values of the visible image.
2. The image fusing method of claim 1 , wherein the first reference value is obtained by adding the mean to a standard deviation of the luminance values of the thermal image.
3. The image fusing method of claim 1 , wherein the second reference value is obtained by subtracting a standard deviation from the mean of the luminance values of the visible image.
4. The image fusing method of claim 1 , wherein the third reference value is preset at an image fusing processor performing the fusing.
5. The image fusing method of claim 1 , wherein the setting the pixel coefficient for each pixel of the thermal image comprises setting a pixel coefficient for a pixel of the thermal image to a value “1” if this pixel is determined to be the important pixel of the thermal image, and a corresponding pixel of the visible image is determined to be not the first important pixel of the visible image but the second important pixel of the visible image, as a result of the determining.
6. The image fusing method of claim 1 , wherein the setting the pixel coefficient for each pixel of the thermal image comprises setting a pixel coefficient for a pixel of the thermal image to a value selected between “0” and “1” if a corresponding pixel of the visible image is determined to be not the second important pixel of the visible image but the first important pixel of the visible image, as a result of the determining.
7. The image fusing method of claim 6 , wherein in the setting the pixel coefficient for each pixel of the thermal image, a pixel coefficient for a pixel of the thermal image set if this pixel is determined to be not the important pixel of the thermal image is smaller than a pixel coefficient for this pixel of the thermal image set if this pixel is determined to be the important pixel of the thermal image and the corresponding pixel of the visible image is determined to be not the second important pixel of the visible image but the first important pixel of the visible image, as a result of the determining.
8. The image fusing method of claim 1 , wherein the fusing the other thermal image and the visible image is performed by adding the visible image, to which a weight is applied, and the other thermal image to which the pixel coefficient and another weight are applied.
9. The image fusing method of claim 1 , further comprising adjusting a luminance value of each pixel of an original thermal image of the subject to generate the thermal image before the detecting the important pixel of the thermal image,
wherein the adjusting comprises:
increasing a luminance value of a pixel of the original thermal image if the luminance value of this pixel is higher than a mean of luminance values of the visible image; and
decreasing the luminance value of this pixel if the luminance value of this pixel is not higher than the mean of the luminance values of the visible image.
10. An image fusing apparatus for fusing a thermal image and a visible image of a subject, the image fusing apparatus comprising:
a hardware processor configured to implement:
a first determination processor configured to detect an important pixel of the thermal image based on a luminance value of each pixel of the thermal image;
a second determination processor configured to detect a first important pixel and a second important pixel of the visible image based on a luminance value of each pixel of the visible image;
a pixel coefficient setting processor configured to set a pixel coefficient for each pixel of the thermal image according to whether each pixel of the thermal image is the important pixel and whether each pixel of the visible image is at least one of the first important pixel and the second important pixel;
a thermal image processor configured to generate another thermal image by applying the pixel coefficient to the luminance value of each pixel of the thermal image; and
an image fusing processor configured to fuse the other thermal image and the visible image to generate a fused image,
wherein the first determination processor is configured to determine the important pixel of the thermal image of which a luminance value is equal to or greater than a first reference value calculated based on a mean of luminance values of the thermal image, and
wherein the second determination processor is configured to determine the first important pixel of the visible image of which a luminance value is equal to or greater than a second reference value calculated based on a mean of luminance values of the visible image, and determine the second important pixel of the visible.
11. The image fusing apparatus of claim 10 , wherein the first reference value is obtained by adding the mean to a standard deviation of the luminance values of the thermal image.
12. The image fusing apparatus of claim 10 , wherein the second reference value is obtained by subtracting a standard deviation from the mean of the luminance values of the visible image.
13. The image fusing apparatus of claim 10 , wherein the third reference value is preset at the image fusing processor.
14. The image fusing apparatus of claim 10 , wherein the pixel coefficient setting processor sets a pixel coefficient for a pixel of the thermal image to a value “1” if this pixel is determined to be the important pixel of the thermal image, and a corresponding pixel of the visible image is determined to be not the first important pixel of the visible image but the second important pixel of the visible image, as a result of the determining by the first and second determination processors.
15. The image fusing apparatus of claim 10 , wherein the pixel coefficient setting processor sets a pixel coefficient for a pixel of the thermal image to a value selected between “0” and “1” if a corresponding pixel of the visible image is determined to be not the second important pixel of the visible image but the first important pixel of the visible image, as a result of the determining by the first and second determination processors.
16. The image fusing apparatus of claim 15 , wherein the pixel coefficient setting processor sets the pixel coefficient for each pixel of the thermal image such that a pixel coefficient for a pixel of the thermal image set if this pixel is determined to be not the important pixel of the thermal image is smaller than a pixel coefficient for this pixel of the thermal image set if this pixel is determined to be the important pixel of the thermal image and the corresponding pixel of the visible image is determined to be not the second important pixel of the visible image but the first important pixel of the visible image, as a result of the determining by the first and second determination processors.
17. The image fusing apparatus of claim 10 , wherein the image fusing processor is configured to fuse the other thermal image and the visible image by adding the visible image, to which a weight is applied, and the other thermal image to which the pixel coefficient and another weight are applied.
18. The image fusing apparatus of claim 10 , wherein the thermal image processor is further configured to adjust a luminance value of each pixel of an original thermal image of the subject to generate the thermal image before the determination processor detects the important pixel of the thermal image, and
wherein the thermal image processor adjusts the luminance value of each pixel of the original thermal image such that a luminance value of a pixel of the original thermal image is increased if the luminance value of this pixel is higher than a mean of luminance values of the visible image, and the luminance value of this pixel is decreased if the luminance value of this pixel is not higher than the mean of the luminance values of the visible image.