Image processing device, image processing method and imaging device

1. An image processing device, comprising:

circuitry configured to:

set a first modulation coefficient used for correction of infrared luminance information of an infrared image obtained in a state in which infrared light is radiated, wherein

the first modulation coefficient is set based on a first ratio between the infrared luminance information and visible luminance information of a visible image obtained in a state in which the infrared light is not radiated;

correct the first modulation coefficient to a second modulation coefficient based on a parameter indicating brightness of a subject of the visible image or brightness of a photographing environment; and

correct the infrared luminance information based on the second modulation coefficient.

2. The image processing device according to claim 1 , wherein

the circuitry is further configured to calculate the second modulation coefficient based on synthesis of the first modulation coefficient with a third modulation coefficient in a case in which the infrared luminance information is not corrected,

the synthesis of the first modulation coefficient with the third modulation coefficient is based on a second ratio, and

the second ratio is calculated based on the parameter.

3. The image processing device according to claim 1 , wherein the circuitry is further configured to convert the first modulation coefficient into the second modulation coefficient based on a first non-linear function in a case in which the brightness of the subject of the visible image or the brightness of the photographing environment is determined to be less than a threshold value.

4. The image processing device according to claim 1 , wherein

the circuitry is further configured to calculate the second modulation coefficient based on synthesis of a fourth modulation coefficient with a fifth modulation coefficient,

the fourth modulation coefficient is obtained based on conversion of the first modulation coefficient through a first linear function,

the fifth modulation coefficient is obtained based on conversion of the first modulation coefficient through a second non-linear function,

the synthesis of the fourth modulation coefficient with the fifth modulation coefficient is based on a third ratio, and

the third ratio is calculated based on the parameter.

5. The image processing device according to claim 4 , wherein the circuitry is further configured to select a combination of the first linear function and the second non-linear function from among a plurality of functions based on the parameter.

6. The image processing device according to claim 5 , wherein the plurality of functions includes a non-linear function and at least one of a linear function in which the fourth modulation coefficient is equal to the first modulation coefficient and a linear function in which the fourth modulation coefficient is 1.

7. The image processing device according to claim 1 , wherein the circuitry is further configured to limit a lower limit value of the second modulation coefficient based on the parameter in a case in which the brightness of the subject of the visible image or the brightness of the photographing environment is determined to be less than a threshold value.

8. The image processing device according to claim 1 , wherein the circuitry is further configured to limit an upper limit value of the second modulation coefficient based on the parameter in a case in which the visible luminance information is larger than the infrared luminance information.

9. The image processing device according to claim 1 , wherein the parameter is a gain adjusted by exposure control for the visible image.

10. The image processing device according to claim 1 , wherein the parameter is a measured value of the brightness of the subject of the visible image or the brightness of the photographing environment.

11. The image processing device according to claim 1 , wherein the circuitry is further configured to set the first modulation coefficient based on a ratio between a low frequency component of the infrared luminance information and a low frequency component of the visible luminance information.

12. The image processing device according to claim 1 , wherein the circuitry is further configured to convert the corrected infrared luminance information and chrominance information of the visible image into color information of a color image.

13. The image processing device according to claim 12 , wherein the circuitry is further configured to:

correct the chrominance information based on the second modulation coefficient; and

convert the corrected infrared luminance information and the corrected chrominance information into the color information of the color image.

14. The image processing device according to claim 1 , wherein the circuitry is further configured to:

extract the infrared luminance information from the infrared image; and

extract the visible luminance information from the visible image.

15. An image processing method, comprising:

setting a first modulation coefficient used for correction of infrared luminance information of an infrared image obtained in a state in which infrared light is radiated, wherein

the first modulation coefficient is set based on a ratio between the infrared luminance information and visible luminance information of a visible image obtained in a state in which the infrared light is not radiated;

correcting the first modulation coefficient to a second modulation coefficient based on a parameter indicating brightness of a subject of the visible image or brightness of a photographing environment; and

correcting the infrared luminance information based on the second modulation coefficient.

16. A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations, the operations comprising:

setting a first modulation coefficient used for correction of infrared luminance information of an infrared image obtained in a state in which infrared light is radiated, wherein

the first modulation coefficient is set based on a ratio between the infrared luminance information and visible luminance information of a visible image obtained in a state in which the infrared light is not radiated;

correcting the first modulation coefficient to a second modulation coefficient based on a parameter indicating brightness of a subject of the visible image or brightness of a photographing environment; and

correcting the infrared luminance information based on the second modulation coefficient.

17. An imaging device, comprising:

an imaging element;

an infrared light irradiation control unit configured to control turn on or off of irradiation of an imaging range with infrared light; and

circuitry configured to:

set a first modulation coefficient used for correction of infrared luminance information of an infrared image obtained in a state in which infrared light is radiated, wherein

the first modulation coefficient is set based on a ratio between the infrared luminance information and visible luminance information of a visible image obtained in a state in which the infrared light is not radiated;

correct the first modulation coefficient to a second modulation coefficient based on a parameter indicating brightness of a subject of the visible image or brightness of a photographing environment; and

correct the infrared luminance information based on the second modulation coefficient.

18. The imaging device according to claim 17 , wherein

the imaging element is configured to:

divide one frame period corresponding to a frame rate into three or more sub frame periods;

sweep pixel values corresponding to an entire screen through a focal plane reading system at intervals of the three or more sub frame periods; and

generate sub frame images which are equal in number to a number of sub frame periods in the one frame period, and

the infrared light irradiation control unit is further configured to control the turn on or off of the irradiation of the imaging range with the infrared light in units of a same time length unit as a sub frame period of the three or more sub frame periods in the one frame period.

19. The imaging device according to claim 18 , further comprising:

an infrared image generating unit configured to generate the infrared image from a plurality of sub frame images in which a period in which the infrared light is radiated is included in an exposure time; and

a visible image generating unit configured to generate the visible image from a plurality of sub frame images in which a period in which the infrared light is radiated is not included in an exposure time.

20. The imaging device according to claim 17 , wherein the circuitry is further configured to convert the corrected infrared luminance information and chrominance information of the visible image into color information of a color image.