Image processing apparatus, image capturing apparatus, and storage medium that stores image processing program
An image processing apparatus generates a first image by performing a resolution enhancement process for the input image, and a second image by performing a noise reduction process for a noise reduction target image. In the noise reduction process, the image processing apparatus extracts a first partial image containing a target pixel and a plurality of second partial images containing a reference pixel, calculates a correlation value between the first partial image and the plurality of second partial images, provides a weight to each of the plurality of second partial images based on a characteristic of the resolution enhancement process and the correlation value, calculates a pixel value of the target pixel using a pixel value of the reference pixel in the plurality of second partial images and the weight, and generate the second image using the calculated pixel value.
1. An image processing apparatus configured to perform image processing for an input image generated by image capturing through an image capturing optical system, the image processing apparatus comprising:
a first processor configured to generate a first image by performing a resolution enhancement process for the input image; and
a second processor configured to generate a second image by performing a noise reduction process for a noise reduction target image, which is the first image or an image generated from the first image,
wherein in the noise reduction process, the second processor is configured to:
extract a first partial image containing a target pixel and a plurality of second partial images containing a reference pixel, from the noise reduction target image;
calculate a correlation value between the first partial image and the plurality of second partial images;
provide a weight to each of the plurality of second partial images based on a characteristic of the resolution enhancement process and the correlation value; and
calculate a pixel value of the target pixel using a pixel value of the reference pixel in the plurality of second partial images and the weight, and generate the second image using the calculated pixel value.
2. The image processing apparatus according to claim 1 , further comprising a noise acquirer configured to acquire a first noise amount contained in the input image,
wherein in the noise reduction process, the second processor is configured to:
acquire a second noise amount contained in the first image using the first noise amount and the characteristic of resolution enhancement process; and
provide the weight to each of the plurality of second partial images according to the second noise amount and the correlation value.
3. The image processing apparatus according to claim 2 , wherein in the noise reduction process, the second processor is configured to provide the weight to each of the plurality of second partial images based on the correlation value and the second noise amount in the target pixel.
4. The image processing apparatus according to claim 2 , wherein in the noise reduction process, the second processor is configured to provide the weight to each of the plurality of second partial images based on the correlation value and a mean value or a median of the second noise amount in the first partial image.
5. The image processing apparatus according to claim 2 , wherein in the noise reduction process, the second processor is configured to provide the weight to each of the plurality of second partial images based on the correlation value and the second noise amount in the first and second partial images.
6. The image processing apparatus according to claim 2 , wherein in the noise reduction process, the second processor is configured to set the number of pixels for the first partial image and the number of pixels for the second partial image based on the second noise amount in the target pixel.
7. The image processing apparatus according to claim 1 , wherein in the noise reduction process, the second processor is configured to extract the plurality of second partial images from an extractable region set in the noise reduction target image based on the characteristic of the resolution enhancement process.
8. The image processing apparatus according to claim 1 , wherein the first processor performs the resolution enhancement process based on an optical characteristic of the image capturing optical system.
9. The image processing apparatus according to claim 1 , wherein the first processor performs the resolution enhancement process based on an optical characteristic of the image capturing optical system which is different for each region in the input image.
10. The image processing apparatus according to claim 1 , wherein the characteristic of the resolution enhancement process is a frequency characteristic based on an optical characteristic of the image capturing optical system or a characteristic relating to a noise gain by the resolution enhancement process.
11. An image capturing apparatus comprising:
an image capturer configured to generate a captured image through image capturing by an image capturing optical system; and
an image processing apparatus configured to perform image processing for the captured image as an input image,
wherein the image processing apparatus includes:
a first processor configured to generate a first image by performing a resolution enhancement process for the input image; and
a second processor configured to generate a second image by performing a noise reduction process for a noise reduction target image, which is the first image or an image generated from the first image,
wherein in the noise reduction process, the second processor is configured to:
extract a first partial image containing a target pixel and a plurality of second partial images containing a reference pixel, from the noise reduction target image;
calculate a correlation value between the first partial image and the plurality of second partial images;
provide a weight to each of the plurality of second partial images based on a characteristic of the resolution enhancement process and the correlation value; and
calculate a pixel value of the target pixel using a pixel value of the reference pixel in the plurality of second partial images and the weight, and generate the second image using the calculated pixel value.
12. A non-transitory computer readable storage medium configured to store an image processing program that enables a computer to perform image processing for an input image generated by image capturing through an image capturing optical system, the image processing comprising:
a first process configured to generate a first image by performing a resolution enhancement process for the input image; and
a second process configured to generate a second image by performing a noise reduction process for a noise reduction target image, which is the first image or an image generated from the first image,
wherein in the noise reduction process, the second process is configured to:
extract a first partial image containing a target pixel and a plurality of second partial images containing a reference pixel, from the noise reduction target image;
calculate a correlation value between the first partial image and the plurality of second partial images;
provide a weight to each of the plurality of second partial images based on a characteristic of the resolution enhancement process and the correlation value; and
calculate a pixel value of the target pixel using a pixel value of the reference pixel in the plurality of second partial images and the weight, and generate the second image using the calculated pixel value.