Image processing apparatus, method, and recording medium
A blur characteristic of a captured image changes due to demosaicing at the time of blur correction of an image pickup optical system. As a result, favorable blur correction cannot be performed. Pieces of RAW data are input, correction coefficients for correcting a blur of RAW images represented by the pieces of RAW data are obtained with respect to a plurality of colors, the blur of the RAW images represented by the pieces of RAW data is corrected on the basis of the correction coefficients obtained with respect to the plurality of colors, corrected images are obtained, and a demosaic process is performed on the plurality of obtained corrected images, thereby generating output image data.
1. An image processing apparatus comprising:
a memory;
a processor coupled to the memory which executes the following:
inputting RAW data, the RAW data being obtained by sampling light that enters an image pickup apparatus including a color filter comprising a plurality of colors and a sensor, the sampling being performed in the sensor via the color filters;
generating color planes each corresponding to one of the plurality of colors by using an arrangement of the color filter, values at pixel positions which do not correspond to the corresponding color of the color plane being zero;
correcting a blur of each of color planes to obtain a corrected color plane by
G c ″=m c ×[R c *{m c ×( h c *f c )}]
c: color of color plane
G c ″: Corrected color plane for color c
m c : mask function for color c (1 at positions for color c and 0 at positions for colors other than color c)
R c : image recovery filter for color c
h c :PSF corresponds to color c
f c :color plane for color c;
and
performing a demosaic process on the corrected color planes, thereby generating output image data.
2. The image processing apparatus according to claim 1 , further comprising obtaining the correction coefficient;
wherein the correcting the blur of the RAW images represented by the RAW data on the basis of the correction coefficients obtained.
3. The image processing apparatus according to claim 2 , wherein the correcting corrects the blur of the RAW images using image recovery filters that are generated using the correction coefficients, thereby obtaining the corrected images.
4. The image processing apparatus according to claim 1 , wherein a linear or nonlinear demosaic process is performed on the plurality of corrected images.
5. The image processing apparatus according to claim 1 , wherein the correction increases a frequency response in a Nyquist frequency of the RAW images represented by the RAW data with respect to the plurality of colors.
6. The image processing apparatus according to claim 2 , wherein the correction coefficients are determined on the basis of an optical characteristic of the image pickup apparatus.
7. An image processing method comprising:
inputting RAW data, the RAW data being obtained by sampling light that enters an image pickup apparatus including a color filter comprising a plurality of colors and a sensor, the sampling being performed in the sensor via the color filters;
generating color planes each corresponding to one of the plurality of colors by using an arrangement of the color filter, values at pixel positions which do not correspond to the corresponding color of the color plane being zero;
correcting a blur of each of color planes to obtain a corrected color plane by
G c ″=m c ×[R c *{m c ×( h c *f c )}]
c: color of color plane
G c ″: Corrected color plane for color c
m c : mask function for color c (1 at positions for color c and 0 at positions for colors other than color c)
R c : image recovery filter for color c
h c :PSF corresponds to color c
f c :color plane for color c;
and
performing a demosaic process on the corrected color planes, thereby generating output image data.
8. A non-transitory computer-readable recording medium storing a program causing a computer to function as:
input means for inputting RAW data, the RAW data being obtained by sampling light that enters an image pickup apparatus including a color filter comprising a plurality of colors and a sensor, the sampling being performed in the sensor via the color filters;
generating means for generating color planes each corresponding to one of the plurality of colors by using an arrangement of the color filter, values at pixel positions which do not correspond to the corresponding color of the color plane being zero;
correcting means for correcting a blur of each of color planes to obtain a corrected color plane by
G c ″=m c ×[R c *{m c ×( h c *f c )}]
c: color of color plane
G c ″: Corrected color plane for color c
m c : mask function for color c (1 at positions for color c and 0 at positions for colors other than color c)
R c : image recovery filter for color c
h c :PSF corresponds to color c
f c :color plane for color c;
and
demosaicing means for performing a demosaic process on the corrected color planes corrected by the correcting means, thereby generating output image data.