Super resolution system and method with database-free texture synthesis
A super resolution (SR) method or system with database-free texture synthesis is disclosed. An input image is up-sampled to result in an up-sampled image. It is determined whether the input image possesses a smooth region. Edges of the up-sampled image are enhanced, and the enhancing edges step is bypassed if the smooth region has been detected. The enhanced or unenhanced up-sampled image is texture synthesized by taking the input image as texture example, thereby resulting in a synthesized image.
1. A super resolution (SR) system with database-free texture synthesis using a processor, the system comprising:
an up-sampling unit configured to up-sample an input image, thereby resulting in an up-sampled image;
an edge enhancement unit configured to enhance edges of the up-sampled image;
a smooth region detection unit configured to determine whether the input image possesses a smooth region, wherein the edge enhancement unit is bypassed if the smooth region has been detected;
a texture synthesis unit configured to take the input image as texture example to perform texture synthesis on the enhanced up-sampled image from the edge enhancement unit or on the unenhanced up-sampled image from the up-sampling unit, thereby resulting in a synthesized image; and
a refinement unit configured to refine the synthesized image to restrict the synthesized image to be faithful to the input image;
wherein the refinement unit applies reconstruction constraint to the synthesized image with an energy function:
E ( I h |I l )=| D ( I h *G )− I l | 2
where I l is the input image, I h is the up-sampled image, G is a Gaussian kernel and D is a down-sample process.
2. The system of claim 1 , wherein the up-sampling unit performs double interpolation on the input image.
3. The system of claim 1 , wherein the edge enhancement unit performs edge enhancement on the up-sampled image by increasing high-frequency part of the up-sampled image.
4. The system of claim 1 , wherein the smooth region detection unit performs detection on the input image according to the following steps:
marking a pixel of the input image as the smooth region if a difference between the input image and an image interpolated from the input image is smaller than a predetermined first threshold; and
determining that the input image possesses the smooth region when a number of the marked pixels is greater than a predetermined second threshold.
5. The system of claim 1 , wherein the texture synthesis unit searches a best match pixel in a search window of the input image for each pixel in the enhanced or unenhanced up-sampled image.
6. The system of claim 5 , wherein the best match pixel q in the input image I l for the pixel p in the enhanced or unenhanced up-sampled image I h is expressed as follows:
BM
p
=
arg
min
q
∑
m
∈
Ω
p
∑
n
∈
Ω
q
I
n
l
-
I
m
h
2
,
q
∈
N
p
where Ω p and Ω q are correlation regions of p and q respectively.
7. The system of claim 5 , wherein the search window is substantially smaller than an entire size of the input image.
8. The system of claim 5 , further comprising a texture extraction unit configured to extract high-frequency part of the input image.
9. The system of claim 8 , wherein the texture extraction unit extracts the high-frequency part according to the following steps:
obtaining low-frequency part of the input image; and
subtracting the low-frequency part from the input image, thereby obtaining the high-frequency part of the input image.
10. The system of claim 8 , wherein the texture synthesis unit adds the extracted high-frequency part into the enhanced or unenhanced up-sampled image, thereby resulting in the synthesized image.
11. The system of claim 1 , wherein the refinement unit minimizes the energy function via gradient descent:
I t+1 h =I t h τ∇E ( I t h )
where t is an iteration index, and τ is a step size.
12. A super resolution (SR) method with database-free texture synthesis, comprising a processor for performing the following steps:
up-sampling an input image to result in an up-sampled image;
determining whether the input image possesses a smooth region;
enhancing edges of the up-sampled image, wherein the enhancing edges step is bypassed if the smooth region has been detected;
texture synthesizing the enhanced or unenhanced up-sampled image by taking the input image as texture example, thereby resulting in a synthesized image; and
refining the synthesized image to restrict the synthesized image to be faithful to the input image;
wherein the step of refining the synthesized image comprises applying reconstruction constraint to the synthesized image with an energy function:
E ( I h |I l )=| D ( I h *G )− I l | 2
where I l is the input image, I h is the up-sampled image, G is a Gaussian kernel and D is a down-sample process.
13. The method of claim 12 , wherein the up-sampling step performs double interpolation on the input image.
14. The method of claim 12 , wherein the enhancing edges step comprises:
increasing high-frequency part of the up-sampled image.
15. The method of claim 12 , wherein the determining step comprises:
marking a pixel of the input image as the smooth region if a difference between the input image and an image interpolated from the input image is smaller than a predetermined first threshold; and
determining that the input image possesses the smooth region when a number of the marked pixels is greater than a predetermined second threshold.
16. The method of claim 12 , wherein the texture synthesizing step comprises:
searching a best match pixel in a search window of the input image for each pixel in the enhanced or unenhanced up-sampled image.
17. The method of claim 16 , wherein the best match pixel q in the input image I l for the pixel p in the enhanced or unenhanced up-sampled image I h is expressed as follows:
BM
p
=
arg
min
q
∑
m
∈
Ω
p
∑
n
∈
Ω
q
I
n
l
-
I
m
h
2
,
q
∈
N
p
where Ω p and Ω q are correlation regions of p and q respectively.
18. The method of claim 16 , wherein the search window is substantially smaller than an entire size of the input image.
19. The method of claim 16 , wherein the texture synthesizing step further comprises a step of extracting high-frequency part of the input image.
20. The method of claim 19 , wherein the high-frequency part is extracted according to the following steps:
obtaining low-frequency part of the input image; and
subtracting the low-frequency part from the input image, thereby obtaining the high-frequency part of the input image.
21. The method of claim 19 , wherein the texture synthesizing step further comprises a step of adding the extracted high-frequency part into the enhanced or unenhanced up-sampled image, thereby resulting in the synthesized image.
22. The method of claim 12 , wherein the energy function is minimized via gradient descent:
I t+1 h =I t h −τ∇E ( I t h )
where t is an iteration index, and τ is a step size.