Image processing apparatus, image processing method, and program

1. An image processing apparatus comprising:

an image processing unit generating a pixel value of an output image by carrying out a pixel value blending process on a long exposure image and a short exposure image,

wherein the image processing unit includes:

an exposure compensation unit carrying out exposure compensation of the long exposure image and the short exposure image in keeping with an exposure ratio between long exposure and short exposure;

a blur information measuring unit using the long exposure image and the short exposure image after the exposure compensation to calculate blur information that is an index value showing a degree of discrepancy between pixel values at a corresponding pixel position in the long exposure image and the short exposure image;

a blending coefficient calculating unit using the blur information to decide a blending coefficient as a blending ratio to be used in the pixel value blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image; and

a blending unit carrying out the pixel value blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image using the blending coefficient calculated by the blending coefficient calculating unit to decide the pixel value of the output image.

2. The image processing apparatus according to claim 1 ,

wherein the image processing unit inputs one image in which long exposure pixels and short exposure pixels are mixed, and further includes:

a sensitivity-specific interpolation unit carrying out an interpolation process on an input image to generate a long exposure image in which all pixels are set at pixel values of long exposure pixels and a short exposure image in which all pixels are set at pixel values of short exposure pixels; and

an exposure compensation unit carrying out exposure compensation in keeping with an exposure ratio between the long exposure pixels and the short exposure pixels on the long exposure image and the short exposure image outputted by the sensitivity-specific interpolation unit, and

the blur information measuring unit calculates the blur information using the exposure-compensated long exposure image and the exposure-compensated short exposure image generated by the exposure compensation unit.

3. The image processing apparatus according to claim 2 ,

wherein the blending coefficient calculating unit uses the blur information to decide the blending coefficient as the blending ratio to be used in the pixel value blending process on the pixel values at the corresponding pixel position in the exposure-compensated long exposure image and the exposure-compensated short exposure image.

4. The image processing apparatus according to claim 2 ,

wherein the blending unit decides the pixel value of the output image by carrying out the pixel value blending process on the pixel values at the corresponding pixel position in the exposure-compensated long exposure image and the exposure-compensated short exposure image using the blending coefficient calculated by the blending coefficient calculating unit.

5. The image processing apparatus according to claim 1 ,

wherein the blur information measuring unit calculates the blur information for each color using the long exposure image and the short exposure image that have been subjected to the exposure compensation and outputs a maximum value out of the blur information for each color at each pixel position as the blur information corresponding to each pixel.

6. The image processing apparatus according to claim 1 ,

wherein the blur information measuring unit includes a noise estimating unit estimating noise included in the long exposure image and the short exposure image and calculates the blur information corresponding to a pixel by subtracting the noise estimated by the noise estimating unit from a value calculated based on a difference in pixel values of corresponding pixels in the long exposure image and the short exposure image that have been subjected to the exposure compensation.

7. The image processing apparatus according to claim 1 ,

wherein for pixels where a pixel value in the long exposure image is at a saturation level, the blending coefficient calculating unit calculates a blending coefficient that suppresses the blending ratio of the pixel value of the long exposure image.

8. The image processing apparatus according to claim 1 ,

wherein the blending unit carries out the pixel value blending process on the pixel values at corresponding pixel positions in the long exposure image and the short exposure image to generate a high dynamic range (HDR) image as the output image.

9. An image processing method executed by an image processing apparatus, comprising:

executing, with an image processing unit, image processing that generates a pixel value of an output image by carrying out a pixel value blending process on a long exposure image and a short exposure image,

wherein the image processing includes:

carrying out exposure compensation of the long exposure image and the short exposure image in keeping with an exposure ratio between long exposure and short exposure;

using the long exposure image and the short exposure image after the exposure compensation to calculate blur information that is an index value showing a degree of discrepancy between pixel values at a corresponding pixel position in the long exposure image and the short exposure image;

using the blur information to decide a blending coefficient as a blending ratio to be used in the pixel value blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image; and

carrying out the pixel value blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image using the calculated blending coefficient to decide the pixel value of the output image.

10. A method comprising:

using an image processing unit to generate a pixel value of an output image by carrying out a pixel value blending process on a long exposure image and a short exposure image;

using an exposure compensation unit to carry out exposure compensation of the long exposure image and the short exposure image in keeping with an exposure ratio between long exposure and short exposure;

using a blur information measuring unit to use the long exposure image and the short exposure image after the exposure compensation to calculate blur information that is an index value showing a degree of discrepancy between pixel values at a corresponding pixel position in the long exposure image and the short exposure image;

using a blending coefficient calculating unit to use the blur information to decide a blending coefficient as a blending ratio to be used in the pixel value blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image; and

using a blending unit to carry out the pixel value blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image using the blending coefficient calculated by the blending coefficient calculating unit to decide the pixel value of the output image.

11. A method according to claim 10 , further comprising:

using the image processing unit to input one image in which long exposure pixels and short exposure pixels are mixed;

using a sensitivity-specific interpolation unit to carry out an interpolation process on an input image to generate a long exposure image in which all pixels are set at pixel values of long exposure pixels and a short exposure image in which all pixels are set at pixel values of short exposure pixels;

using an exposure compensation unit to carry out exposure compensation in keeping with an exposure ratio between the long exposure pixels and the short exposure pixels on the long exposure image and the short exposure image outputted by the sensitivity-specific interpolation unit; and

using the blur information measuring unit to calculate the blur information using the exposure-compensated long exposure image and the exposure-compensated short exposure image generated by the exposure compensation unit.

12. A method according to claim 10 , further comprising:

using the blending coefficient to calculating unit to use the blur information to decide the blending coefficient as the blending ratio to be used in the pixel value blending process on the pixel values at the corresponding pixel position in the exposure-compensated long exposure image and the exposure-compensated short exposure image.

13. A method according to claim 10 , further comprising:

using the blending unit to decide the pixel value of the output image by carrying out the pixel value blending process on the pixel values at the corresponding pixel position in the exposure-compensated long exposure image and the exposure-compensated short exposure image using the blending coefficient calculated by the blending coefficient calculating unit.

14. A method according to claim 10 , further comprising:

using the blur information measuring unit to calculate the blur information for each color using the long exposure image and the short exposure image that have been subjected to the exposure compensation and outputs a maximum value out of the blur information for each color at each pixel position as the blur information corresponding to each pixel.

15. A method according to claim 10 , further comprising:

using the blur information measuring unit to include a noise estimating unit estimating noise included in the long exposure image and the short exposure image and calculates the blur information corresponding to a pixel by subtracting the noise estimated by the noise estimating unit from a value calculated based on a difference in pixel values of corresponding pixels in the long exposure image and the short exposure image that have been subjected to the exposure compensation.

16. A method according to claim 10 , further comprising:

using the blending unit to carry out the pixel value blending process on the pixel values at corresponding pixel positions in the long exposure image and the short exposure image to generate a high dynamic range (HDR) image as the output image.