Image processing apparatus and method, recording medium, and program

1. An image processing apparatus for generating a color image, in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity, on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, comprising:

extraction means for extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

generation means for making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted by the extraction means and generating local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

edge detection means for detecting an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

first interpolation means for interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component, on the basis of the direction of the edge detected by the edge detection means;

statistic-information computing means for computing statistical information on the basis of the pixels included in the local area information; and

second interpolation means for interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the statistical information, wherein

the statistic-information computing means computes, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

wherein the second interpolation means interpolates the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing means.

2. The image processing apparatus according to claim 1 , further comprising:

gamma conversion means for performing gamma conversion of the pixels included in the local area information; and

inverse gamma conversion means for performing inverse gamma conversion of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the color component other than the first color component associated with the pixel of interest, which is interpolated by the second interpolation means.

3. The image processing apparatus according to claim 1 , wherein the first color component is a color component that statistically has the highest signal level of the plurality of color components.

4. The image processing apparatus according to claim 1 , wherein the first color component is a color component that occupies the largest portion of the color mosaic image of the plurality of color components.

5. The image processing apparatus according to claim 1 , further comprising defective-pixel interpolation means for interpolating a defective pixel included in the local area information using pixels neighboring the defective pixel.

6. An image processing apparatus for generating a color image, in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity, on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, comprising:

extraction means for extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

generation means for making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted by the extraction means and generating local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

edge detection means for detecting an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

first interpolation means for interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component, on the basis of the direction of the edge detected by the edge detection means;

statistic-information computing means for computing statistical information on the basis of the pixels included in the local area information; and

second interpolation means for interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the statistical information, wherein

the statistic-information computing means computes, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

wherein the second interpolation means includes

first computation means for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing means; and

second computation means for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of the other color component, which is computed by the statistic-information computing means,

wherein one of the first computation means and the second computation means is selected to interpolate the other color component associated with the pixel of interest.

7. The image processing apparatus according to claim 6 , wherein the second interpolation means selects one of the first computation means and the second computation means on the basis of the standard deviation of the first color component, which is computed by the statistic-information computing means, to interpolate the other color component associated with the pixel of interest.

8. The image processing apparatus according to claim 6 , further comprising:

gamma conversion means for performing gamma conversion of the pixels included in the local area information; and

inverse gamma conversion means for performing inverse gamma conversion of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the color component other than the first color component associated with the pixel of interest, which is interpolated by the second interpolation means.

9. The image processing apparatus according to claim 6 , wherein the first color component is a color component that statistically has the highest signal level of the plurality of color components.

10. The image processing apparatus according to claim 6 , wherein the first color component is a color component that occupies the largest portion of the color mosaic image of the plurality of color components.

11. The image processing apparatus according to claim 6 , further comprising defective-pixel interpolation means for interpolating a defective pixel included in the local area information using pixels neighboring the defective pixel.

12. An image processing apparatus for generating a color image in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, comprising:

extraction unit configured to extract a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

generation unit configured to make uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted by the extraction unit and configured to generate local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

edge detection unit configured to detect an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

first interpolation unit configured to interpolate the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component, on the basis of the direction of the edge detected by the edge detection unit;

statistic-information computing unit configured to compute statistic information on the basis of the pixels included in the local area information; and

second interpolation unit configured to interpolate a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation unit, and the statistic information, wherein

the statistic-information computing unit is configured to compute, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

the second interpolation unit is configured to interpolate the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation unit, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing unit.

13. An image processing apparatus for generating a color image in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, comprising:

extraction unit configured to extract a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

generation unit configured to make uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted by the extraction unit and configured to generate local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

edge detection unit configured to detect an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

first interpolation unit configured to interpolate the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component, on the basis of the direction of the edge detected by the edge detection unit;

statistic-information computing unit configured to compute statistic information on the basis of the pixels included in the local area information; and

second interpolation unit configured to interpolate a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation unit, and the statistic information, wherein

the statistic-information computing unit is configured to compute, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

the second interpolation unit includes

first computation unit configured to compute the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation unit, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing unit; and

second computation unit for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation unit, and the average of the other color component, which is computed by the statistic-information computing unit,

wherein one of the first computation unit and the second computation unit is selected to interpolate the other color component associated with the pixel of interest.

14. An image processing method for generating a color image in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the method comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistical information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistical information, wherein

the statistic-information computing step computes, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

the second interpolation step interpolates the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation step, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing step.

15. An image processing method for generating a color image in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the method comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistical information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistical information, wherein

the statistic-information computing step computes, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

the second interpolation step includes

a first computation step for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation step, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing step; and

a second computation step for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation step, and the average of the other color component, which is computed by the statistic-information computing step,

wherein one of the first computation step and the second computation step is selected to interpolate the other color component associated with the pixel of interest.

16. A computer-readable medium including computer-executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for generating a color image in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the program comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistic information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistic information, wherein

the statistic-information computing step computes, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

the second interpolation step interpolates the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation step, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing step.

17. A computer-readable medium including computer-executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for generating a color image in which each pixel in the color image has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the program comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each of the pixels having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, those pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, those pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistic information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistic information, wherein

the statistic-information computing step computes, as the statistical information, at least one of an average of each color component, a standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of those pixels included in the local area information, and

the second interpolation step includes

a first computation step for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation step, and the average of the other color component, the standard deviation of the other color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing step; and

a second computation step for computing the other color component, the color component other than the first color component associated with the pixel of interest, on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation step, and the average of the other color component, which is computed by the statistic-information computing step,

wherein one of the first computation step and the second computation step is selected to interpolate the other color component associated with the pixel of interest.