Image processing device, imaging device, and imaging method
An information processing device that includes an exposure control unit alternately repeats a first frame and a second frame. An infrared light irradiation control unit performs irradiation of infrared light in a predetermined infrared light irradiation period which is equal to or less than an aggregate period of the first frame and the second frame. An image signal acquisition unit acquires a first image signal which is the image signal in the first frame and a second image signal which is the image signal in the second frame. An extraction unit extracts a visible light intensity per unit time and an infrared light intensity per unit time from the first image signal and the second image signal. A generation unit generates an image signal corresponding to visible light and an image signal corresponding to the infrared light.
1. An image processing device, comprising:
an exposure control unit configured to:
control timing from exposure start to exposure stop of an image sensor which outputs an image signal of a subject; and
alternately repeat a first frame in which an exposure time is set to a first exposure time and a second frame in which the exposure time is set to a second exposure time longer than the first exposure time, wherein
a difference between a time of start of the second frame and a reset time in the second frame, is smaller than a difference between a time of start of the first frame and a reset time in the first frame;
an infrared light irradiation control unit configured to irradiate the subject with infrared light in an infrared light irradiation period, wherein the infrared light irradiation period is one of shorter than or equal to an aggregate period of the first frame and the second frame;
an image signal acquisition unit configured to acquire a first image signal which is an image signal in the first frame, and a second image signal which is an image signal in the second frame;
an extraction unit configured to extract a visible light intensity per unit time and an infrared light intensity per unit time from the first image signal and the second image signal; and
a generation unit configured to generate an image signal corresponding to visible light and an image signal corresponding to the infrared light based on the extracted visible light intensity per unit time and the extracted infrared light intensity per unit time.
2. The image processing device according to claim 1 , wherein
a period in which an exposure period overlaps with the infrared light irradiation period in the first frame is a first infrared light irradiation period,
a period in which the exposure period overlaps with the infrared light irradiation period in the second frame is a second infrared light irradiation period, and
a ratio of the first exposure time to the first infrared light irradiation period is different from a ratio of the second exposure time to the second infrared light irradiation period.
3. The image processing device according to claim 1 , further comprising:
a conversion unit configured to convert the image signal corresponding to the visible light into a luminance signal and a color signal; and
a correction unit configured to correct the luminance signal and the color signal, based on the image signal corresponding to the infrared light.
4. The image processing device according to claim 1 , further comprising a separation unit, wherein
the image signal of the subject includes an image signal corresponding to red light and the infrared light, an image signal corresponding to green light and the infrared light, and an image signal corresponding to blue light and the infrared light, and
the separation unit is configured to separate the image signal corresponding to the infrared light from each of the image signal corresponding to the red light and the infrared light, the image signal corresponding to the green light and the infrared light, and the image signal corresponding to the blue light and the infrared light to obtain
an image signal corresponding to the red light,
an image signal corresponding to the green light, and
an image signal corresponding to the blue light.
5. The image processing device according to claim 4 , wherein
the image sensor further includes an image signal corresponding to white light and the infrared light, and
the separation unit is further configured to separate the image signal corresponding to the infrared light from the image signal corresponding to the white light and the infrared light to obtain an image signal corresponding to the white light.
6. An imaging device, comprising:
an image processing device that includes:
an image sensor configured to output an image signal of a subject;
an infrared light irradiation unit configured to irradiate the subject with infrared light;
an exposure control unit configured to:
control timing from exposure start to exposure stop of the image sensor; and
alternately repeat a first frame in which an exposure time is set to a first exposure time and a second frame in which the exposure time is set to a second exposure time longer than the first exposure time, wherein
a difference between a time of start of the second frame and a reset time in the second frame, is smaller than a difference between a time of start of the first frame and a reset time in the first frame;
an infrared light irradiation control unit configured to control the infrared light irradiation unit to irradiate the subject with the infrared light in an infrared light irradiation period, wherein the infrared light irradiation period is one of shorter than or equal to an aggregate period of the first frame and the second frame;
an image signal acquisition unit configured to acquire a first image signal which is an image signal in the first frame, and a second image signal which is an image signal in the second frame;
an extraction unit configured to extract a visible light intensity per unit time and an infrared light intensity per unit time from the first image signal and the second image signal; and
a generation unit configured to generate an image signal corresponding to visible light and an image signal corresponding to the infrared light based on the extracted visible light intensity per unit time and the extracted infrared light intensity per unit time.
7. An imaging method, comprising:
controlling timing from exposure start to exposure stop of an image sensor which outputs an image signal of a subject;
alternately repeating a first frame in which an exposure time is set to a first exposure time and a second frame in which the exposure time is set to a second exposure time longer than the first exposure time, wherein
a difference between a time of start of the second frame and a reset time in the second frame, is smaller than a difference between a time of start of the first frame and a reset time in the first frame;
irradiating the subject with infrared light in an infrared light irradiation period, wherein the infrared light irradiation period is one of shorter than or equal to an aggregate period of the first frame and the second frame;
acquiring a first image signal which is an image signal in the first frame, and a second image signal which is an image signal in the second frame;
extracting a visible light intensity per unit time and an infrared light intensity per unit time from the first image signal and the second image signal; and
generating an image signal corresponding to visible light and an image signal corresponding to the infrared light based on the extracted visible light intensity per unit time and the extracted infrared light intensity per unit time.
8. The image processing device according to claim 1 , wherein the infrared light irradiation period is a period in which the first frame is switched to the second frame.
9. The image processing device according to claim 1 , wherein the first exposure time is between the reset time in the first frame and a reading time in the first frame.