Image processing apparatus, image pickup apparatus, image processing method, and storage medium
An image processing apparatus includes a separation unit configured to separate luminance information of each pixel in a processing area of an input image into an intrusion component and an object component, a determination unit configured to determine whether each of intrusion components is a first component or a second component, a processing unit configured to generate first information based on the first component, and an image generating unit configured to generate an output image based on the first information, the second component, and the object component.
1. An image processing method comprising the steps of:
separating luminance information of each pixel in a processing area of an input image into a specular reflection light component and a diffused reflection light component;
determining whether each of the specular reflection light components is a first component or a second component;
generating first information by one of:
lowering a resolution of a background image corresponding to the first component;
reducing a luminance value of the background image corresponding to the first component; or
replacing a background image corresponding to the first component with another image; and
generating an output image based on the first information, the second component, and the diffused reflection light component.
2. The image processing method according to claim 1 , further comprising the step of:
selecting part of the input image as the processing area,
wherein the selecting selects an area including an eye detected from the input image as the processing area.
3. The image processing method according to claim 1 , wherein the determining determines whether each of the specular reflection light components is the first component or the second component, using the luminance information of the respective specular reflection light component.
4. The image processing method according to claim 1 , wherein:
the luminance information includes a luminance value each of the specular reflection light components and
the determining determines that each of the specular reflection light intrusion component is:
the first component in a state where the luminance value is smaller than a predetermined value; and
the second component in a state where the luminance value is larger than the predetermined value.
5. The image processing method according to claim 1 , wherein the determining determines whether each of the specular reflection light components is the first component or the second component, using color information of the respective specular reflection light component.
6. The image processing method according to claim 1 , wherein the determining determines whether each of the specular reflection light components is the first component or the second component, using a shape of the respective specular reflection light component.
7. The image processing method according to claim 1 , further comprising the step of:
calculating polarization information from a plurality of polarization images acquired by a plurality of polarization elements arranged in different directions,
wherein the separating separates the diffused reflection light component, which is determined not to depend on directions of the plurality of polarization elements, and the specular reflection light component, which is different from the diffused reflection light component.
8. The image processing method according to claim 1 , wherein:
the generating generates the first information by reducing the luminance value of the background image corresponding to the first component, and
the determining determines whether each of the specular reflection light components is the first component or the second component based on a predetermined value of the luminance value.
9. The image processing method according to claim 1 , wherein:
the generating generates the first information by replacing a background image corresponding to the first component with another image, and
the method further comprises generating the another image using a smoothing filter.
10. An image processing apparatus comprising:
one or more memories storing instructions; and
one or more processors or circuits configured to:
separate luminance information of each pixel in a processing area in an input image into a specular reflection light component and a diffused reflection light component;
determining whether each of the specular reflection light components is a first component or a second component;
generating first information by one of:
lowering a resolution of a background image corresponding to the first component;
reducing a luminance value of the background image corresponding to the first component; or
replacing a background image corresponding to the first component with another image; and
generating an output image based on the first information, the second component, and the diffused reflection light component.
11. A non-transitory computer-readable storage medium storing a program executable by a computer to execute an image processing method comprising the steps of:
separating luminance information of each pixel in a processing area of an input image into a specular reflection light component and a diffused reflection light component;
determining whether each of the specular reflection light components is a first component or a second component;
generating first information by one of:
lowering a resolution of a background image corresponding to the first component;
reducing a luminance value of the background image corresponding to the first component; or
replacing a background image corresponding to the first component with another image; and
generating an output image based on the first information, the second component, and the diffused reflection light component.