Image processing apparatus, image processing method, and imaging apparatus

1. An image processing apparatus, comprising:

a demosaic processing unit configured to:

process, based on demosaic processing, a mosaic image having a first number of colors;

generate a first multispectral image based on the processed mosaic image, wherein the first multispectral image has the first number of colors;

generate a second multispectral image based on the processed mosaic image, wherein the second multispectral image has a second number of colors equal to or more than three but less than the first number of colors; and

generate a luminance signal based on application of a specific filter to the mosaic image;

a linear matrix processing unit configured to apply linear matrix processing to each of the first multispectral image and the second multispectral image to generate a first image and a second image, wherein the first image has a third number of colors less than the second number of colors and the second image has the third number of colors; and

a combination processing unit configured to combine the first image and the second image based on the luminance signal.

2. The image processing apparatus according to claim 1 , wherein the demosaic processing unit is further configured to:

calculate a local average value in each pixel of the mosaic image for each color of the mosaic image; and

interpolate a pixel value in each pixel based on the local average value.

3. The image processing apparatus according to claim 2 , wherein the demosaic processing unit is further configured to calculate the local average value for each color of the mosaic image based on average of a plurality of colors in the first number of colors so as to reduce the first number of colors to the second number of colors.

4. The image processing apparatus according to claim 3 , wherein a correlation of spectral sensitivity between the plurality of colors is higher than a correlation of spectral sensitivity between other colors included in the first number of colors.

5. The image processing apparatus according to claim 3 , wherein the demosaic processing unit is further configured to generate the first multispectral image and the second multispectral image based on the luminance signal the mosaic image.

6. The image processing apparatus according to claim 5 , wherein the demosaic processing unit is further configured to calculate the local average value for each color of the mosaic image based on the luminance signal.

7. The image processing apparatus according to claim 6 , wherein the demosaic processing unit is further configured to:

obtain edge information based on edge detection of the luminance signal; and

calculate the local average value based on the edge information.

8. The image processing apparatus according to claim 1 , wherein the combination processing unit is further configured to blend the first image and the second image based on a blend coefficient.

9. The image processing apparatus according to claim 8 , wherein the combination processing unit is further configured to determine the blend coefficient based on a saturation of the first image and a saturation of the second image.

10. The image processing apparatus according to claim 1 , wherein

the demosaic processing unit is further configured to obtain edge information based on edge detection of the luminance signal, and

the combination processing unit is further configured to determine a blend coefficient based on the edge information.

11. The image processing apparatus according to claim 1 , wherein the specific filter is a box filter of 2×2 pixels.

12. An image processing method, comprising:

processing, by demosaic processing, a mosaic image having a first number of colors;

generating a first multispectral image based on the processed mosaic image, wherein the first multispectral image has the first number of color;

generating a second multispectral image based on the processed mosaic image, wherein the second multispectral image has a second number of colors equal to or more than three but less than the first number of colors;

generating a luminance signal based on application of a specific filter to the mosaic image;

applying linear matrix processing to each of the first multispectral image and the second multispectral image to generate a first image and a second image, wherein the first image has a third number of colors less than the second number of colors and the second image has the third number of colors; and

combining the first image and the second image based on the luminance signal.

13. An imaging apparatus, comprising:

an imaging unit includes a color filter in which one of a plurality of kinds of filters has different spectral characteristics, wherein

the color filter is allocated to each pixel position, and

the imaging unit is configured to acquire a mosaic image having a first number of colors that is same as a number of kinds of the filters; and

a demosaic processing unit configured to:

process, based on demosaic processing, a mosaic image having the first number of colors,

generate a first multispectral image based on the processed mosaic image, wherein the first multispectral image has the first number of colors, and

generate a second multispectral image based on the processed mosaic image, wherein the second multispectral image has a second number of colors equal to or more than three but less than the first number of colors;

generate a luminance signal based on application of a specific filter to the mosaic image;

a linear matrix processing unit configured to apply linear matrix processing to each of the first multispectral image and the second multispectral image to generate a first image and a second image, wherein the first image has a third number of colors less than the second number of colors and the second image has the third number of colors; and

a combination processing unit configured to combine the first image and the second image based on the luminance signal.