Image processing apparatus and image restoration method and program
An image processing apparatus is provided which performs image restoration as necessary while a computing processing burden based on an image restoration algorithm is reduced. RAW data of a blurred image is input to an RGB interpolation section. The RGB interpolation section generates an R component image, a G component image, and a B component image based on the RAW data. The G component image is input to an image restoration processing section and the R and B component images are input to an RGB to YCC section. The image restoration processing section performs an image restoration process based on a predetermined image restoration algorithm by using a PSF image representing a predetermined point spread function. The RGB to YCC section generates a brightness component image Y and color difference component images CR and CB based on the G component image having been subjected to the image restoration process and the R and B component images which was not subjected to any image restoration process.
1. An image processing apparatus comprising:
an image separation section for separating a plurality of color component images forming a blurred image which is captured in accordance with a contribution level to a brightness component into a first image and a second image having a smaller contribution level to the brightness component than the first image;
an image restoration processing section for performing an image restoration process with regard to each of the first and the second images based on a predetermined image restoration algorithm including an iterative computation using a PSF image representing a point spread function which is defined by an image capturing condition and iteratively processing the image, at least once, until a difference between the image after any processing and the immediately preceding image is less than a predetermined threshold value; wherein the iterative computation performs computation on any portion of a particular image and repeats the computation process on all or substantially all of the previously computed processed portion;
a parameter setting section for setting parameters of the iterative computation including the threshold value: and
a restored image generating section for generating a restored image corresponding to the blurred image based on the first and the second images having been subjected to the image restoration process; wherein
the parameter setting section sets the threshold value with regard to the second image greater than the threshold value with regard to the first image.
2. The image processing apparatus according to claim 1 , wherein the image restoration processing section does not perform the image restoration process with regard to the second image based on the image restoration algorithm if a size of the PSF image is smaller than or equal to a predetermined size, and performs the image restoration process also with regard to the second image based on the image restoration algorithm if the size of the PSF image is larger than the predetermined size.
3. The image processing apparatus according to claim 2 , wherein:
the image restoration algorithm is an algorithm using the iterative computation with a steepest descent method, in which if the blurred image is G, the restored image is F and the PSF image is H, the image restoration process is performed by calculating an evaluation value J based on a size of a difference between an image HF obtained by causing the PSF image H to operate on the restored image F, and the blurred image G, and iteratively performing a process of calculating the restored image F and a process of calculating the evaluation value J until a size of a gradient of the evaluation value J becomes smaller than or equal to a threshold; and
the threshold with regard to the second image is larger than the threshold with regard to the first image.
4. The image processing apparatus according to claim 1 , wherein:
the image restoration processing section extracts an edge portion from the second image and performs the image restoration process with regard to the extracted edge portion; and
the restored image generating section generates the restored image corresponding to the blurred image based on the first image having been subjected to the image restoration process and the second image having been subjected to the image restoration process with regard to the edge portion.
5. The image processing apparatus according to claim 1 , wherein:
the image restoration processing section performs a resolution conversion on the second image so that a resolution of the second image becomes lower than a resolution of the first image, and performs the image restoration process with regard to the second image having been subjected to the resolution conversion; and
the restored image generating section generates the restored image corresponding to the blurred image based on the first image having been subjected to the image restoration process and the second image having been subjected to the image restoration process after the resolution conversion.
6. The image processing apparatus according to claim 1 , wherein the image restoration processing section corrects the PSF image so that a center of gravity of the PSF image is located at a center of a particular pixel forming the PSF image, and uses the corrected PSF image to perform the image restoration process.
7. The image processing apparatus according to claim 1 , further comprising a pixel interpolation section for performing only a pixel interpolation which is a linear interpolation with regard to the blurred image before the image restoration processing section performs the image restoration process.
8. The image processing apparatus according to claim 1 , wherein the first image is a G component image, and the second image is an R component image and a B component image.
9. The image processing apparatus according to claim 1 , wherein the first image is a brightness component image, and the second image is a color difference component image.
10. A program for causing a computer to function as an image processing apparatus which generates a restored image from a blurred image which is captured, based on a predetermined image restoration algorithm using a degradation function which is defined by an image capturing condition, the program causing the computer to function as:
an image separation section for separating a plurality of color component images forming a blurred image which is captured in accordance with a contribution level to a brightness component into a first image and a second image having a smaller contribution level to the brightness component than the first image;
an image restoration processing section for performing an image restoration process with regard to each of the first and the second images based on a predetermined image restoration algorithm including an iterative computation using a PSF image representing a point spread function which is defined by an image capturing condition and iteratively processing the image, at least once, until a difference between the image after any processing and the immediately preceding image is less than a predetermined threshold value; wherein the iterative computation performs computation on any portion of a particular image and repeats the computation process on all or substantially all of the previously computed processed portion;
a parameter setting section for setting parameters of the iterative computation including the threshold value: and
a restored image generating section for generating a restored image corresponding to the blurred image based on the first and the second images having been subjected to the image restoration process; wherein
the parameter setting section sets the threshold value with regard to the second image greater than the threshold value with regard to the first image.
11. The program according to claim 10 , wherein the image restoration processing section corrects the PSF image so that a center of gravity of the PSF image is located at a center of a particular pixel forming the PSF image, and uses the corrected PSF image to perform the image restoration process.