Image processing method and apparatus with image deblurring
An image processing method includes computing a blur kernel of an input image using a kernel estimation model, performing kernel-based deblurring on the input image using the blur kernel to obtain a deconvolved image, and generating an output image by performing kernel-free deblurring based on the deconvolved image.
1 . An image processing method, comprising:
obtaining an input image;
deblurring the input image to obtain a temporary sharp image;
generating first image data and second image data based on the input image and the temporary sharp image, respectively;
computing a blur kernel for the input image based on the first image data and the second image data using a kernel estimation model by transforming the input image, transforming the temporary sharp image, and combining the transformed input image with the transformed temporary sharp image;
performing a primary deblurring on the input image using the blur kernel to obtain a deconvolved image; and
generating an output image by performing a secondary deblurring based on the deconvolved image.
2 . The image processing method of claim 1 , wherein
the secondary deblurring comprises image processing using a deep learning model without genera-tion of kernels by the deep learning model.
3 . The image processing method of claim 1 , wherein
computing the blur kernel comprises:
transforming the input image into a first frequency image in a frequency domain through a logarithmic Fourier transform;
transforming the temporary sharp image into a second frequency image in the frequency domain; and
generating first input data for computing the blur kernel by combining the first frequency image and the second frequency image.
4 . The image processing method of claim 3 , wherein:
the blur kernel represents a blur characteristic of the input image in the frequency domain, and
the performing the primary deblurring on the input image using the blur kernel to obtain the deconvolved image comprises:
generating a third frequency image by subtracting the blur kernel from the first frequency image; and
transforming the third frequency image into the deconvolved image in a spatial domain.
5 . The image processing method of claim 1 , wherein:
the computing the blur kernel comprises:
transforming the input image into a first frequency image in a frequency domain through a logarithmic Fourier transform; and
computing the blur kernel based on the first frequency image using the kernel estimation model.
6 . The image processing method of claim 5 , wherein:
the blur kernel represents a blur characteristic of the input image in the frequency domain, and
the performing the primary deblurring on the input image using the blur kernel to obtain the deconvolved image comprises:
generating a third frequency image by subtracting the blur kernel from the first frequency image; and
transforming the third frequency image into the deconvolved image in a spatial domain.
7 . The image processing method of claim 1 , wherein the generating the output image comprises:
computing a compensation image based on the input image and the deconvolved image using a deblurring model; and
generating the output image by adding the compensation image to the deconvolved image.
8 . The image processing method of claim 7 , wherein:
the compensation image compensating for at least one of a first error and a second error, wherein the first error is based on the computing the blur kernel and the second error is based on using a non-uniform blur in the input image as a uniform blur.
9 . The image processing method of claim 1 , wherein:
the kernel estimation model is a neural network model.
10 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:
compute a blur kernel of an input image using a kernel estimation model;
transform the input image into a first frequency image in a frequency domain;
generate a second frequency image by subtracting the blur kernel from the first frequency image;
perform a primary deblurring based on the second frequency image to obtain a deconvolved image; and
generate an output image by performing a secondary deblurring based on the deconvolved image using a neural network.
11 . An image processing apparatus, comprising:
a processor; and
a memory configured to store instructions executable by the processor, wherein, in response to the instructions being executed by the processor, the processor performs operations including:
obtaining an input image;
performing an initial deblurring on the input image to obtain a temporary sharp image;
generating first image data and second image data based on the input image and the temporary sharp image, respectively;
computing a blur kernel for the input image using a kernel estimation model based on the first image data and the second image data,
performing a primary deblurring on the input image using the blur kernel to obtain a deconvolved image,
and generating an output image by performing a secondary deblurring based on the deconvolved image.
12 . The image processing apparatus of claim 11 , wherein the initial deblurring is performed using a neural network.
13 . The image processing apparatus of claim 11 , wherein the processor further performs operations including:
transforming the input image into a first frequency image in a frequency domain through a logarithmic Fourier transform,
transforming the temporary sharp image into a second frequency image in the frequency domain, and
generating first input data by combining the first frequency image and the second frequency image.
14 . The image processing apparatus of claim 13 , wherein:
the blur kernel represents a blur characteristic of the input image in the frequency domain, and
wherein the processor further performs operations including:
generating a third frequency image by subtracting the blur kernel from the first frequency image, and
transforming the third frequency image into the deconvolved image in a spatial domain.
15 . The image processing apparatus of claim 11 , wherein the processor further performs operations including:
transforming the input image into a first frequency image in a frequency domain through a logarithmic Fourier transform, and
computing the blur kernel based on the first frequency image using the kernel estimation model.
16 . The image processing apparatus of claim 11 , wherein the processor further performs operations including:
computing a compensation image based on the input image and the deconvolved image using a deblurring model, and
generating the output image by adding the compensation image to the deconvolved image.
17 . The image processing apparatus of claim 16 , wherein:
the compensation image compensating for at least one of a first error and a second error, wherein the first error is based on computing the blur kernel and the second error is based on using a non-uniform blur in the input image as a uniform blur.
18 . An electronic device, comprising:
a camera configured to generate an input image; and
a processor configured to:
generate a temporary sharp image by performing a temporary deblurring on the input image;
generate first input data based on the input image and the temporary sharp image;
compute a blur kernel of the input image based on the first input data using a kernel estimation model;
generate a deconvolved image by performing a primary deblurring on the input image using the blur kernel; and
generate an output image by performing a secondary deblurring based on the deconvolved image.
19 . The electronic device of claim 18 , wherein the processor is further configured to:
transform the input image into a first frequency image in a frequency domain through a logarithmic Fourier transform;
transform the temporary sharp image into a second frequency image in the frequency domain; and
generate the first input data by combining the first frequency image and the second frequency image.