Image processing device, image processing method, and recording medium
An image processing device includes: an estimation unit configured to estimate a spectral reflectance ratio at each pixel position in a visible light image and a near infrared image in which an object of photographing is photographed, the spectral reflectance ratio indicating a ratio of a spectral reflectance of a visible light wavelength region to a spectral reflectance of a near infrared light wavelength region; and a restoration unit configured to multiply an estimated spectral reflectance ratio by luminance of the near infrared image at each pixel position and generate a restored visible light image.
1. An image processing device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
estimate a spectral reflectance ratio at each pixel position in a visible light image and a near infrared image in which an object of photographing is photographed, the spectral reflectance ratio indicating a ratio of a spectral reflectance of a visible light wavelength region to a spectral reflectance of a near infrared light wavelength region; and
multiply the estimated spectral reflectance ratio by luminance of the near infrared image at each pixel position and generate a restored visible light image.
2. The image processing device according to claim 1 , wherein the at least one processor is to execute the instructions to further:
receive input of the visible light image and the estimated spectral reflectance ratio, and calculate a corrected spectral reflectance ratio by correcting the spectral reflectance ratio of the object of photographing at each pixel of interest by use of information of the visible light image and the spectral reflectance ratio in a neighborhood around the pixel of interest, wherein
the at least one processor multiplies the corrected spectral reflectance ratio by luminance of the near infrared image and generates the restored visible light image.
3. The image processing device according to claim 2 , wherein
the at least one processor extracts pixel groups from neighborhoods around the pixel of interest, which are regarded as a same region, by using the information of the visible light image, and corrects the spectral reflectance ratio of the pixel of interest based on the spectral reflectance ratios of the pixel groups extracted.
4. The image processing device according to claim 2 , wherein
the at least one processor calculates a composite weight for a pixel in a predetermined neighborhood including the pixel of interest, and calculates the corrected spectral reflectance ratio by combining the spectral reflectance ratios of the pixels in the neighborhood.
5. The image processing device according to claim 1 , wherein the at least one processor is to execute the instructions to further:
estimate an atmospheric transmittance at each pixel of the near infrared image, and correct the near infrared image by using the estimated atmospheric transmittance, wherein
the at last one processor multiplies the estimated spectral reflectance ratio by luminance of a corrected near infrared image, which is corrected by the second correction unit, and generates the restored visible light image.
6. The image processing device according to claim 5 , wherein
the at least one processor corrects a pixel value of each the pixel of interest in the near infrared image by using a pixel value of a pixel in a neighborhood around the pixel of interest in the near infrared image.
7. The image processing device according to claim 1 , wherein the at least one processor is to execute the instructions to further:
correct the near infrared image by using the visible light image, wherein
the at least one processor multiplies the estimated spectral reflectance ratio by luminance of a corrected near infrared image, which is corrected by the third correction unit, and generates the restored visible light image.
8. The image processing device according to claim 7 , wherein
in a case that, at a predetermined pixel in the near infrared image and the visible light image, the atmospheric transmittance of the near infrared light is lower than that of visible light, a prediction is made that there is an error in estimating the atmospheric transmittance.
9. An image processing method comprising:
estimating a spectral reflectance ratio at each pixel position in a visible light image and a near infrared image in which an object of photographing is photographed, the spectral reflectance ratio indicating a ratio of a spectral reflectance of a visible light wavelength region to a spectral reflectance of a near infrared light wavelength region; and
multiplying the estimated spectral reflectance ratio by luminance of the near infrared image at each pixel position and generating a restored visible light image.
10. A non-transitory computer-readable recording medium which stores a program making a computer execute:
estimating a spectral reflectance ratio at each pixel position in a visible light image and a near infrared image in which an object of photographing is photographed, the spectral reflectance ratio indicating a ratio of a spectral reflectance of a visible light wavelength region to a spectral reflectance of a near infrared light wavelength region; and
multiplying the estimated spectral reflectance ratio by luminance of the near infrared image at each pixel position and generating a restored visible light image.