Single image super-resolution method using transform-invariant directional total variation with S½+L½-norm
A super-resolution method for generating a high-resolution (HR) image from a low-resolution (LR) blurred image is provided. The method is based on a transform-invariant directional total variation (TI-DTV) approach with Schatten p=1/2 (S 1/2 -norm) and L 1/2 -norm penalties. The S 1/2 -norm and the L 1/2 -norm are used to induce a lower-rank component and a sparse component of the LR blurred image so as to determine an affine transform to be adopted in the TI-DTV approach. In particular, the affine transform is determined such that a weighted sum of the S 1/2 -norm and the L 1/2 -norm is substantially minimized. Based on the alternating direction method of multipliers (ADMM), an iterative algorithm is developed to determine the affine transform. The determined affine transform is used to transform a candidate HR image to a transformed image used in computing a directional total variation (DTV), which is involved in determining the HR image.
1. A method for generating a high-resolution image from a low-resolution blurred image by one or more processors, the method comprising:
determining an affine transform for transforming a first image into a second image that is aligned horizontally and vertically with the first image, the affine transform being determined from the low-resolution blurred image such that a weighted sum of a Schatten p=1/2 -norm (S 1/2 -norm) of a low-rank component and a L 1/2 -norm of a sparse component is substantially minimized;
wherein the low-rank component and the spare component are obtained by applying a candidate affine transform to the low-resolution blurred image to yield a third image and then decomposing the third image into the low-rank component for representing an edge structure of the third image, and the spare component for representing noise or blur information thereof;
wherein the determining of the affine transform comprises iteratively refining the candidate affine transform so as to successively reduce the weighted sum,
wherein the weighted sum is given by
∥ A∥ S 1/2 1/2 +λ∥E∥ 1/2 1/2
where:
A and E are the low-rank component and the sparse component, respectively,
∥A∥ S 1/2 1/2 is the S 1/2 -norm of A,
∥E∥ 1/2 1/2 is the L 1/2 -norm of E, and
λ is a pre-determined parameter for balancing a contribution of the L 1/2 -norm with the weighted sum;
wherein the candidate affine transform is iteratively refined according to
{
A
k
+
1
=
arg
min
A
1
2
A
-
(
Io
τ
-
E
k
+
1
μ
Y
k
)
F
2
+
A
S
1
/
2
1
/
2
E
k
+
1
=
arg
min
E
1
2
λ
E
-
(
Io
τ
-
A
k
+
1
+
1
μ
Y
k
)
F
2
+
E
l
a
a
Y
k
+
1
=
Y
k
+
μ
(
Io
τ
-
A
k
+
1
-
E
k
+
1
)
where:
Ioτ=A k +E k ,
A k is the low-rank component obtained in a k th iteration, and A k+1 is the low-rank component obtained in a (k+1)th iteration, an iteration immediately after the k th iteration,
E k is the spare component obtained in the k th iteration, and E k+1 is the spare component obtained in the (k+1)th iteration,
Y k is an augmented Lagrangian multiplier obtained in the k th iteration, and Y k+1 is an augmented Lagrangian multiplier obtained in the (k+1)th iteration,
μ is a nonnegative parameter,
∥A S 1/2 1/2 denotes a S 1/2 -norm, and
∥E∥ l a a denotes a L 1/2 -norm;
wherein the candidate affine transform is iteratively refined until
Io
τ
-
A
k
+
1
-
E
k
+
1
F
Io
τ
F
<
ɛ
is satisfied, where ε is a pre-determined threshold;
wherein the iterative refinement of the candidate affine transform is computed by an iterative process comprising:
Step 1: performing an initialization comprising:
K= 0; A 0 =0; Y 0 =0; E 0 =0;Δτ 0 >0;μ 0 >0;ρ>1;
Step 2: determining:
Ioτ−E k +μ −1 Y k =U r S r V r T ;
A
k
+
1
=
U
r
H
2
μ
(
S
r
)
V
r
T
Step 3: determining:
E
k
+
1
=
H
2
λ
μ
(
Io
τ
-
A
k
+
1
+
μ
-
k
Y
k
)
Step 4: determining:
Δτ k+1 =(∇ I ) + ( A k+1 +E k+1 −Ioτ−μ −1 Y k )
Step 5: determining:
Y k+1 =Y k +μ( Ioτ−A k+1 −μ −1 Y k+1 )
Step 6: determining:
μ k+1 =ρμ k
Step 7:
if
Io
τ
-
A
k
+
1
-
E
k
+
1
F
Io
τ
F
<
ɛ
for the pre-determined threshold, repeat Step 2;
otherwise set values for A, E and τ; and
generating the high-resolution image by a transform-invariant directional total variation (TI-DTV) regularization method,
wherein the regularization method comprises computing a directional total variation (DTV) of a transformed image obtained by applying the determined affine transform to a candidate high-resolution image.
2. The method of claim 1 , wherein the determining of the high-resolution image comprises iteratively refining the candidate high-resolution image so as to successively reduce a regularization-based cost function.
3. The method of claim 2 , wherein the regularization-based cost function is given by
∥ D ( K{circle around (×)}I′ HR )− I LR ∥ 2 +λ·DTV ( A τ I′L HR )+β· TV ( I′ HR )
where:
I LR is the low-resolution blurred image;
I′ HR is the candidate high-resolution image;
D is a down-sampling operator;
K is a blurring kernel;
λ and β are regularization parameters;
A τ is a linear operator of the affine transform;
TV denotes a total variation; and
DTV denotes a directional total variation given by
DTV
(
I
)
=
∑
x
=
1
N
1
∑
y
=
1
N
2
-
1
(
I
(
x
,
y
+
1
)
)
+
∑
y
=
1
N
2
∑
x
=
1
N
1
-
1
(
I
(
x
+
1
,
y
)
)
for an image IεR N 1 ×N 2 .
4. An image-processing system comprising one or more processors configured to generate a high-resolution image from a low-resolution blurred image by the method of claim 1 .
5. An image-processing system comprising one or more processors configured to generate a high-resolution image from a low-resolution blurred image by the method of claim 2 .
6. An image-processing system comprising one or more processors configured to generate a high-resolution image from a low-resolution blurred image by the method of claim 3 .