Image processing apparatus, image processing method, and electronic apparatus to detect and correct false color in high dynamic range imaging
View Patent ↗According to an embodiment of the present disclosure, a part estimated to have a high degree of generation of false color for predetermined parts of an input image imaged by different exposure times at a predetermined spatial period is detected. A false color intensity value indicating an estimated degree of generation of false color is calculated for each of the predetermined parts. A high dynamic range image is generated by executing processing to merge long-time exposure pixels and short-time exposure pixels making up the input image. False color correction processing is executed according to the intensity of the false color intensity value, to the high dynamic range image. The present disclosure may be applied to, for example, an image processing unit within a solid state image sensor.
1. An image processing apparatus, comprising:
a false color intensity detection unit configured
to detect a part estimated to have a high degree of generation of false color for predetermined parts of an input image imaged by different exposure times at a predetermined spatial period, and
to calculate a false color intensity value indicating an estimated degree of generation of false color for each of the predetermined parts;
a merge processing unit configured to generate a high dynamic range image by executing processing to merge long-time exposure pixels and short-time exposure pixels making up the input image; and
a false color correction processing unit configured to execute false color correction processing according to an intensity of the false color intensity value to the high dynamic range image.
2. The image processing apparatus according to claim 1 , further comprising:
a moving subject detection unit configured to detect a moving subject imaged on the input image from the long-time exposure pixels and the short-time exposure pixels making up the input image, and output a moving subject detection value indicating an extent of a moving subject for each of the predetermined parts of the input image; and
a saturation level detection unit configured to detect a saturation level in the input image, and output a saturation level detection value for each of the predetermined parts of the input image;
in which,
the false color correction processing unit is further configured to refer to the moving subject detection value and the saturation level detection value, to calculate the false color intensity value.
3. The image processing apparatus according to claim 1 , further comprising:
an amplitude level detection unit configured to detect an AC component contained in the input image, and output an amplitude level detection value for each of the predetermined parts of the input image; and
a saturation level detection unit configured to detect a saturation level in the input image, and output a saturation level detection value for each of the predetermined parts of the input image;
in which,
the false color correction processing unit is further configured to refer to the amplitude level detection value and the saturation level detection value, to calculate the false color intensity value.
4. The image processing apparatus according to claim 1 , further comprising:
a moving subject detection unit configured to detect a moving subject imaged on the input image from the long-time exposure pixels and the short-time exposure pixels making up the input image, and output a moving subject detection value indicating an extent of a moving subject for each of the predetermined parts of the input image;
an amplitude level detection unit configured to detect an AC component contained in the input image, and output an amplitude level detection value for each of the predetermined parts of the input image; and
a saturation level detection unit configured to detect a saturation level in the input image, and output a saturation level detection value for each of the predetermined parts of the input image;
in which,
the false color correction processing unit is further configured to refer to the moving subject detection value, the amplitude level detection value, and the saturation level detection value, to calculate the false color intensity value.
5. The image processing apparatus according to claim 4 , further comprising
a composition ratio calculation unit configured to
refer to
the moving subject detection value calculated by the moving subject detection unit,
the amplitude level detection value calculated by the amplitude level detection unit, and
the saturation level detection value calculated by the saturation level detection unit, and
calculate a composition ratio to use in the processing of the merge processing unit to merge the long-time exposure pixels and the short-time exposure pixels making up the input image.
6. The image processing apparatus according to claim 5 , further comprising:
a noise estimate calculation unit configured to calculate, from all the components that make up the input image, a noise estimate for estimating a noise generated in the input image; and
a merged noise estimate calculation unit configured to calculate a merged noise estimate for estimating a noise generated in the high dynamic range image generated by the merge processing unit, by using the noise estimate calculated by the noise estimate calculation unit and the composition ratio calculated by the composition ratio calculation unit.
7. The image processing apparatus according to claim 1 , further comprising:
a reduced image calculation unit configured to refer to the false color intensity value calculated by the false color correction processing unit, and calculate a reduced image of the high dynamic range image being reduced, excluding signal values of predetermined parts of which the false color intensity value is more than or equal to a predetermined value;
and a reduced image storage unit configured to store the reduced image.
8. The image processing apparatus according to claim 7 , further comprising
a reduced image signal interpolation processing unit configured to interpolate pixels that make up the reduced image stored in the reduced image storage unit, in such a manner that the pixels making up the reduced image are invariably at the same distance from a pixel of interest when used in subsequent processing.
9. The image processing apparatus according to claim 8 , further comprising:
a similarity determination unit configured to
execute similarity determination with the use of
pixel values making up the high dynamic range image,
a merged noise estimate calculated by a merged noise estimate calculation unit, and
the reduced image interpolated by the reduced image signal interpolation processing unit, and
calculate a similarity determination value; and
a noise reduction processing unit configured to execute noise reduction processing by mixing the high dynamic range image and reduced image interpolated by the reduced image signal interpolation processing unit, according to an intensity of the similarity determination value calculated by the similarity determination unit.
10. The image processing apparatus according to claim 9 , wherein
the false color correction processing unit is configured to execute false color correction processing by making signal values, of the high dynamic range image after undergoing the noise reduction processing by the noise reduction processing unit, closer to color composition ratios of a corresponding region in the reduced image interpolated by the reduced image signal interpolation processing unit, according to an intensity of the false color intensity value.
11. An image processing method, comprising:
detecting a part estimated to have a high degree of generation of false color for predetermined parts of an input image imaged by different exposure times at a predetermined spatial period, and calculating a false color intensity value indicating an estimated degree of generation of false color for each of the predetermined parts;
generating a high dynamic range image by executing processing to merge long-time exposure pixels and short-time exposure pixels making up the input image; and
executing false color correction processing according to an intensity of the false color intensity value to the high dynamic range image.
12. An electronic apparatus, comprising:
a false color intensity detection unit configured
to detect a part estimated to have a high degree of generation of false color for predetermined parts of an input image imaged by different exposure times at a predetermined spatial period, and
to calculate a false color intensity value indicating an estimated degree of generation of false color for each of the predetermined parts;
a merge processing unit configured to generate a high dynamic range image by executing processing to merge long-time exposure pixels and short-time exposure pixels making up the input image; and
a false color correction processing unit configured to execute false color correction processing according to an intensity of the false color intensity value, to the high dynamic range image.