Image restoration apparatus and method
Provided is an image restoration method and apparatus, in which an image is restored by estimating the blur level of the image. The image restoration apparatus includes a block generation module which generates a plurality of overlapping blocks for an input image; a blur estimation module which generates a plurality of unit step responses for a current overlapping block from the overlapping blocks; a feature-map generation module which generates a feature map for the current overlapping block based on the unit step responses of the current overlapping block; and a filter application module which deblurs the current overlapping block by using a deconvolution filter to filter the feature map of the current overlapping block.
1. An image restoration apparatus comprising:
a block generation module which generates a plurality of overlapping blocks for an input image;
a blur estimation module which generates a plurality of unit step responses for a current overlapping block from the overlapping blocks;
a feature-map generation module which generates a feature map for the current overlapping block based on the unit step responses of the current overlapping block;
a filter application module which deblurs, by a computer processor, the current overlapping block by using a deconvolution filter to filter the feature map of the current overlapping block; and
a blending module which generates a deblurred image of the input image by using a plurality of sub-blocks, the sub-blocks comprising middle portions of the respective overlapping blocks,
wherein the size of each of the overlapping blocks is less than the size of the input image.
2. The image restoration apparatus of claim 1 , further comprising a feature-map refinement module which refines an abnormal coefficient in the feature map of the current overlapping block by using feature maps of overlapping blocks that are adjacent to the current overlapping block.
3. The image restoration apparatus of claim 2 , wherein, if the abnormal coefficient is missing from the feature map of the current overlapping block, the feature-map refinement module refines a coefficient obtained by calculating an average or a distance-weighted average of coefficient values of the feature maps of the adjacent overlapping blocks.
4. The image restoration apparatus of claim 2 , wherein, if the abnormal coefficient is an incorrect coefficient, the feature-map refinement module determines whether differences between a value of the incorrect coefficient and values of coefficients of the feature maps of the adjacent overlapping blocks exceed a predetermined threshold value, and, if the differences exceed the predetermined threshold value, the feature-map refinement module refines the incorrect coefficient by calculating an average of or a distance-weighted average of the coefficient values of the feature maps of the adjacent overlapping blocks.
5. The image restoration apparatus of claim 1 , wherein the blur estimation module detects edges of the current overlapping block, extracts a valid edge from the detected edges, generates a unit step response for each of a plurality of pixels included in the valid edge, and generates a representative step response by superimposing the unit step responses on one another.
6. The image restoration apparatus of claim 5 , wherein the blur estimation module uses one or more Sobel masks that represent different directions.
7. The image restoration apparatus of claim 1 , wherein the feature-map generation module determines a distance between a camera and a subject by comparing a blur level of the current overlapping block with blur levels of a plurality of images captured at various distances from the camera, and then estimates a point spread function (PSF) based on the determined distance.
8. The image restoration apparatus of claim 1 , wherein the feature-map generation module searches a sigma value of a Gaussian blur filter that matches the blur level of the representative step response and generates a Gaussian distribution for the searched signal value.
9. The image restoration apparatus of claim 1 , wherein the feature-map generation module generates a matrix for estimating a PSF based on the blur level of the current overlapping block, and estimates a PSF based on the blur level of the current overlapping block using the matrix.
10. The image restoration apparatus of claim 1 , wherein the deconvolution filter is a Wiener filter or a Lucy-Richardson filter.
11. An image capturing system comprising:
a photographing element which captures an image of a subject;
a driving circuit which adjusts a lens and a diaphragm; and
an image restoration apparatus which deblurs the image captured by the photographing element,
wherein the image restoration apparatus comprises:
a block generation module which generates a plurality of overlapping blocks for an input image;
a blur estimation module which generates a plurality of unit step responses for a current overlapping block from the overlapping blocks;
a feature-map generation module which generates a feature map for the current overlapping block based on the unit step responses of the current overlapping block;
a filter application module which deblurs the current overlapping block by using a deconvolution filter to filter the feature map of the current overlapping block; and
a blending module which generates a deblurred image of the input image by using a plurality of sub-blocks, the sub-blocks comprising middle portions of the respective overlapping blocks,
wherein the size of each of the overlapping blocks is less than the size of the input image.
12. An image restoration method comprising:
generating a plurality of overlapping blocks for an input image;
generating a plurality of unit step responses for a current overlapping block from the overlapping blocks;
generating a feature map for the current overlapping block based on the unit step responses of the current overlapping block;
deblurring the current overlapping block by using a deconvolution filter to filter the feature map of the current overlapping block; and
generating a deblurred image of the input image by using a plurality of sub-blocks, the sub-blocks comprising middle portions of the respective overlapping blocks,
wherein the size of each of the overlapping blocks is less than the size of the input image.
13. The image restoration method of claim 12 , further comprising refining an abnormal coefficient in the feature map of the current overlapping block by using feature maps of overlapping blocks that are adjacent to the current overlapping block.
14. The image restoration method of claim 13 , wherein, if the abnormal coefficient is missing from the feature map of the current overlapping block, the refining the abnormal coefficient comprises refining a coefficient obtained by calculating an average or a distance-weighted average of coefficient values of the feature maps of the adjacent overlapping blocks.
15. The image restoration method of claim 13 , wherein, if the abnormal coefficient is an incorrect coefficient, the refining the abnormal coefficient comprises:
determining whether differences between a value of the incorrect coefficient and values of coefficients of the feature maps of the adjacent overlapping blocks exceed a predetermined threshold; and
if the differences exceed the predetermined threshold value, refining the incorrect coefficient by calculating an average of or a distance-weighted average of the coefficient values of the feature maps of the adjacent overlapping blocks.
16. The image restoration method of claim 12 , wherein the generating the unit responses comprises:
detecting edges of the current overlapping block;
extracting a valid edge from the detected edges;
generating a unit step response for each of a plurality of pixels included in the valid edge; and
generating a representative step response by superimposing the unit step responses on one another.
17. The image restoration method of claim 16 , wherein the detecting the edges of the current overlapping block comprises detecting the edges of the current overlapping block by using one or more Sobel masks that represent different directions.
18. The image restoration method of claim 12 , wherein the generating the feature map comprises:
determining a distance between a camera and a subject by comparing a blur level of the current overlapping block with blur levels of a plurality of images captured at various distances from the camera; and
estimating a point spread function (PSF) based on the determined distance.
19. The image restoration method of claim 12 , wherein the generating the feature map comprises:
searching for a sigma value of a Gaussian blur filter that matches the blur level of the representative step response; and
generating a Gaussian distribution for the searched signal value.
20. The image restoration method of claim 12 , wherein the generating the feature map comprises:
generating a matrix for estimating a PSF based on the blur level of the current overlapping block; and
estimating a PSF based on the blur level of the current overlapping block by using the matrix.
21. The image restoration method of claim 12 , wherein the deconvolution filter is a Wiener filter or a Lucy-Richardson filter.