Image processing device, image processing method, and storage medium

1. An image processing device comprising:

at least one memory storing a set of instructions; and

at least one processor configured to execute the set of instructions to:

receive a visible image of a face;

receive a near-infrared image of the face;

adjust brightness of the visible image based on a frequency distribution of pixel values of the visible image and a frequency distribution of pixel values of the near-infrared image, by adjusting the pixel values of the visible image in such a way that at least a part of a shape of the frequency distribution of the visible image approaches a shape of the frequency distribution of the near-infrared image in a range of the pixel values excluding a first range including a darkest pixel value and a second range image;

specify a relative position at which the visible image is related to the near-infrared image;

invert the adjusted brightness of the visible image;

detect a region of a pupil from a synthetic image obtained by adding up the visible image having the inverted adjusted brightness and the near-infrared image based on the relative position; and

output information on the detected pupil.

2. The image processing device according to claim 1 , wherein

the at least one processor is further configured to execute the instructions to:

detect the face from the visible image;

detect the face from the near-infrared image; and

detect the region of the pupil in a case where the face is detected from both the visible image and the near-infrared image.

3. The image processing device according to claim 1 , wherein

the at least one processor is further configured to execute the instructions to adjust the brightness of the visible image by further:

adjusting the pixel values of the visible image in such a way that a distribution of the pixel values of the visible image approaches a distribution of the pixel values of the near-infrared image.

4. The image processing device according to claim 1 , wherein

the at least one processor is further configured to execute the instructions to adjust the brightness of the visible image by further:

adjusting the pixel values of the visible image in such a way that a pixel value indicated by a frequency peak in a first range from the darkest pixel value in the frequency distribution of the visible image coincides with a pixel value indicating a frequency peak in a second range from the darkest pixel value in the frequency distribution of the near-infrared image.

5. The image processing device according to claim 1 , wherein

the at least one processor is further configured to execute the instructions to

determine wearing of sunglasses based on the pixel values of the pixels in a region including the region of the pupil in the visible image.

6. The image processing device according to claim 1 , wherein

the at least one processor is further configured to execute the instructions to

control intensity of light of a light source that irradiates the face with light based on the pixel values of the pixels in the region including the region of the pupil.

7. An image processing system including the image processing device according to claim 1 , comprising:

a visible imaging device that captures the visible image; and

a near-infrared imaging device that captures the near-infrared image.

8. An image processing method comprising:

receiving, by a processor, a visible image of a face;

receiving, by the processor, a near-infrared image of the face;

adjusting, by the processor, brightness of the visible image based on a frequency distribution of pixel values of the visible image and a frequency distribution of pixel values of the near-infrared image, by adjusting the pixel values of the visible image in such a way that at least a part of a shape of the frequency distribution of the visible image approaches a shape of the frequency distribution of the near-infrared image in a range of the pixel values excluding a first range including a darkest pixel value and a second range including a brightest pixel value of the near-infrared image;

specifying, by the processor, a relative position at which the visible image is related to the near-infrared image;

inverting, by the processor, the adjusted brightness of the visible image;

detecting, by the processor, a region of a pupil from a synthetic image obtained by adding up the visible image having the inverted adjusted brightness and the near-infrared image based on the relative position; and

outputting, by the processor, information on the detected pupil.

9. The image processing method according to claim 8 , further comprising:

detecting, by the processor, the face from the visible image;

detecting, by the processor, the a face from the near-infrared image; and

detecting, by the processor, the region of the pupil in a case where the face is detected from both the visible image and the near-infrared image.

10. The image processing method according to claim 8 , wherein the brightness of the visible image is adjusted by further

adjusting the pixel values of the visible image in such a way that a distribution of the pixel values of the visible image approaches a distribution of the pixel values of the near-infrared image.

11. The image processing method according to claim 8 , wherein the brightness of the visible image is adjusted by further

adjusting the pixel values of the visible image in such a way that a pixel value indicated by a frequency peak in a first range from the darkest pixel value in the frequency distribution of the visible image coincides with a pixel value indicating a frequency peak in a second range from the darkest pixel value in the frequency distribution of the near-infrared image.

12. The image processing method according to claim 8 , further comprising

determining, by the processor, wearing of sunglasses based on the pixel values of the pixels in a region including the region of the pupil in the visible image.

13. The image processing method according to claim 8 , further comprising

controlling, by the processor, intensity of light of a light source that irradiates the face with light based on the pixel values of the pixels in the region including the region of the pupil.

14. A non-transitory computer readable storage medium storing a program that when executed by a computer causes the computer to perform:

first input processing of receiving a visible image of a face;

second input processing for receiving a near-infrared image of the face;

adjustment processing of adjusting brightness of the visible image based on a frequency distribution of pixel values of the visible image and a frequency distribution of pixel values of the near-infrared image, by adjusting the pixel values of the visible image in such a way that at least a part of a shape of the frequency distribution of the visible image approaches a shape of the frequency distribution of the near-infrared image in a range of the pixel values excluding a first range including a darkest pixel value and a second range including a brightest pixel value of the near-infrared image;

alignment processing of specifying a relative position at which the visible image is related to the near-infrared image;

inversion processing of inverting the adjusted brightness of the visible image;

pupil detection processing of detecting a region of a pupil from a synthetic image obtained by adding up the visible image having the inverted adjusted brightness and the near-infrared image based on the relative position; and

output processing of outputting information on the detected pupil.

15. The storage medium according to claim 14 , wherein

the processing further comprises:

first detection processing of detecting the face from the visible image; and

second detection processing of detecting the face from the near-infrared image, wherein

the pupil detection processing detects the region of the pupil in a case where the face is detected from both the visible image and the near-infrared image.

16. The storage medium according to claim 14 , wherein

the adjustment processing further adjusts the pixel values of the visible image in such a way that a distribution of the pixel values of the visible image approaches a distribution of the pixel values of the near-infrared image.

17. The storage medium according to claim 14 , wherein the adjustment processing further adjusts the pixel values of the visible image in such a way that a pixel value indicated by a frequency peak in a first range from the darkest pixel value in the frequency distribution of the visible image coincides with a pixel value indicating a frequency peak in a second range from the darkest pixel value in the frequency distribution of the near-infrared image.