IP Library › Granted Patent US 10,412,324
Granted Patent B2
US 10,412,324 · App. 15/913,945 · Granted Sep 10, 2019

Infrared image reconstruction method based on block-sparse compressive sensing and system thereof

Inventors: Li Kang (Guangdong, CN); Runqing Liang (Guangdong, CN); Jianjun Huang (Guangdong, CN); Jingxiong Huang (Guangdong, CN)
Assignee: SHENZHEN UNIVERSITY
H04N5/33G06T5/50H03M7/3062H04N19/136H04N19/176H04N19/85H04N19/90G06T2207/10048G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,412,324
App. No.
15/913,945
Granted
Sep 10, 2019
Kind
B2
Abstract

The present invention provides an infrared image reconstruction method based on block-sparse compressive sensing. The method includes: a block-sparse representation step: representing consecutive infrared small target image sequences as signals having block-sparse characteristics; a temporal correlation modeling step: modeling a temporal correlation of data between the consecutive infrared small target image sequences under a Bayesian framework; and a signal reconstruction and output step: reconstructing infrared small target images using a sparse learning algorithm and outputting final estimates of the signals. The present invention further provides an infrared image reconstruction system based on block-sparse compressive sensing. According to the technical solution provided in the present invention, in the process of reconstruction by compressive sensing, signals are reconstructed using a correlation of consecutive signals in time, such that the reconstruction precision of a single frame of signal can be improved, and the algorithm running time is significantly reduced.

Claims (538)

1. An infrared image reconstruction method based on block-sparse compressive sensing, comprising:

a block-sparse representation step: representing consecutive infrared small target image sequences as signals having block-sparse characteristics;

a temporal correlation modeling step: modeling a temporal correlation of data between the consecutive infrared small target image sequences under a Bayesian framework; and

a signal reconstruction and output step: reconstructing infrared small target images using a sparse learning algorithm and outputting final estimates of the signals.

2. The infrared image reconstruction method based on block-sparse compressive sensing according to claim 1 , wherein the block-sparse representation step specifically comprises:

representing a multiple-measurement vector model Y=ΦX+V using consecutive infrared small target image sequence signals and a block-sparse matrix, where the infrared small target image signals are x,x∈ M×1 , the block-sparse matrix is X={x 1 , x 2 , . . . , x L }, X∈ M×L , Y [y (.1) , . . . , y (.L) ]∈ N×L is an observation matrix, X [x (.1) , . . . , x (.L) ]∈ M×L , Φ∈ N×M is a known dictionary matrix, V is a noise matrix, y (.i) is an i th column of Y, and x (.i) is an i th column of X; and

convert the multiple-measurement vector model into a single-measurement vector model y=Dx+v, where in the multiple-measurement vector model, the block-sparse matrix X is constituted by consecutive frames x i (i=1, 2, . . . , L) having a temporal correlation, x (i.) (i=1, . . . , M) are independent of each other, x (i.) is an i th row of X, let y=vec(Y T )∈ NL×1 , x=vec(X T )∈ ML×1 , where D=Φ⊗I L , I L is an L-dimension unit matrix, and x is a block-sparse signal and may be represented as

x

=

[

x

1

⁢

⁢

…

⁢

⁢

x

L

︸

x

T

⁡

[

1

]

⁢

⁢

x

2

⁢

⁢

…

⁢

⁢

x

2

⁢

⁢

L

︸

x

T

⁡

[

2

]

⁢

⁢

…

⁢

⁢

x

J

-

L

+

1

⁢

⁢

…

⁢

⁢

x

J

︸

x

T

⁡

[

M

]

]

T

,

where J=M×L.

3. The infrared image reconstruction method based on block-sparse compressive sensing according to claim 2 , wherein the temporal correlation modeling step specifically comprises:

in the multiple-measurement vector model, assuming X i⋅ are independent of each other and comply with a Gaussian distribution: p(X i⋅ ;γ i ,B i )˜N(0,γ i B i ), where a super-parameter γ i controls the sparseness of the vectors X i⋅ , and B i is a positive definite matrix that describes a time correlation structure of X i⋅ and is a parameter to be estimated, assuming a priori of the sparse vector x is p(x;γ i ,B i )˜N(0,Σ 0 ), and modeling a temporal correlation of data between the consecutive infrared small target image sequences under a Bayesian framework, where γ i is a non-negative super-parameter,

Σ

0

=

[

γ

1

⁢

B

1

⋱

γ

M

⁢

B

M

]

,

and the time structure of the signals is described with a prior density for constructing the matrix B i .

4. The infrared image reconstruction method based on block-sparse compressive sensing according to claim 3 , wherein the signal reconstruction and output step specifically comprises:

under the condition that an observation y is known, obtaining a posterior estimate of the sparse vector x using a Bayesian method: p(x|y;σ 2 ,γ i ,B i )˜N(μ x ,Σ x ), where a Gaussian likelihood function of the observation y is:

p

⁡

(

y

|

x

,

σ

2

)

=

(

2

⁢

πσ

2

)

-

N

/

2

⁢

exp

⁡

(

-

1

2

⁢

σ

2

⁢

y

-

Dx

2

)

,

where

μ

x

=

1

σ

2

⁢

Σ

x

⁢

D

T

⁢

y

where and Σ x =Σ 0 −Σ 0 D T (σ 2 I+DΣ 0 D T ) −1 DΣ 0 ; and

estimating the super-parameter in the above equation through a maximum likelihood method or an expectation maximization method:

γ

i

=

Tr

⁡

[

B

-

1

⁡

(

Σ

x

i

+

μ

x

i

⁡

(

μ

x

i

)

T

)

]

L

,

i

=

1

,

…

⁢

,

M

,

⁢

B

=

1

M

⁢

∑

i

=

1

M

⁢

⁢

Σ

x

i

+

μ

x

i

⁡

(

μ

x

i

)

T

γ

i

,

i

=

1

,

…

⁢

,

M

,

⁢

σ

2

=

y

-

D

⁢

⁢

μ

x

2

2

+

σ

′2

⁡

[

ML

-

Tr

⁡

(

Σ

x

⁢

Σ

0

-

1

)

]

NL

,

and outputting final estimates of the signals after the iteration ends.

5. An infrared image reconstruction system based on block-sparse compressive sensing, comprising:

a block-sparse representation module, configured to represent consecutive infrared small target image sequences as signals having block-sparse characteristics;

a temporal correlation modeling module, configured to model a temporal correlation of data between the consecutive infrared small target image sequences under a Bayesian framework; and

a signal reconstruction and output module, configured to reconstruct infrared small target images using a sparse learning algorithm and output final estimates of the signals.

6. The infrared image reconstruction system based on block-sparse compressive sensing according to claim 5 , wherein the block-sparse representation module is specifically configured to:

represent a multiple-measurement vector model Y=ΦX+V using consecutive infrared small target image sequence signals and a block-sparse matrix, where the infrared small target image signals are x,x∈ M×1 , the block-sparse matrix is X={x 1 , x 2 , . . . , x L }, X∈ M×L , Y [y (.1) , . . . , y (.L) ]∈ N×L is an observation matrix, X [x (.1) , . . . , x (.L) ]∈ M×L , Φ∈ N×M is a known dictionary matrix, V is a noise matrix, y (.i) is an i th column of Y, and x (.i) is an i th column of X; and

convert the multiple-measurement vector model into a single-measurement vector model y=Dx+v, where in the multiple-measurement vector model, the block-sparse matrix X is constituted by consecutive frames x i (i=1, 2, . . . , L) having a temporal correlation,) x (i.) (i=1, . . . , M) are independent of each other, x (i.) is an i th row of X, let y=vec(Y T )∈ NL×1 , x=vec(X T )∈ ML×1 , where D=Φ⊗I L , I L is an L-dimension unit matrix, and x is a block-sparse signal and may be represented as

x

=

[

x

1

⁢

⁢

…

⁢

⁢

x

L

︸

x

T

⁡

[

1

]

⁢

⁢

x

2

⁢

⁢

…

⁢

⁢

x

2

⁢

⁢

L

︸

x

T

⁡

[

2

]

⁢

⁢

…

⁢

⁢

x

J

-

L

+

1

⁢

⁢

…

⁢

⁢

x

J

︸

x

T

⁡

[

M

]

]

T

,

where J=M×L.

7. The infrared image reconstruction method based on block-sparse compressive sensing according to claim 6 , wherein the temporal correlation modeling module is specifically configured to:

in the multiple-measurement vector model, assume X i⋅ are independent of each other and comply with a Gaussian distribution: p(X i⋅ ;γ i ,B i )˜N(0,γ i B i ), where a super-parameter γ i controls the sparseness of the vectors X i⋅ , and B i is a positive definite matrix that describes a time correlation structure of X i⋅ and is a parameter to be estimated, assume a priori of the sparse vector x is p(x;γ i ,B i )˜N(0,Σ 0 ), and model a temporal correlation of data between the consecutive infrared small target image sequences under a Bayesian framework, where γ i is a non-negative super-parameter,

Σ

0

=

[

γ

1

⁢

B

1

⋱

γ

M

⁢

B

M

]

,

and the time structure of the signals is described with a prior density for constructing the matrix B i .

8. The infrared image reconstruction method based on block-sparse compressive sensing according to claim 7 , wherein the signal reconstruction and output module is specifically configured to:

under the condition that an observation y is known, obtain a posterior estimate of the sparse vector x using a Bayesian method: p(x|y;σ 2 ,γ i ,B i )˜N(μ x ,Σ x ), where a Gaussian likelihood function of the observation y is:

p

⁡

(

y

|

x

,

σ

2

)

=

(

2

⁢

πσ

2

)

-

N

/

2

⁢

exp

⁡

(

-

1

2

⁢

σ

2

⁢

y

-

Dx

2

)

,

where

μ

x

=

1

σ

2

⁢

Σ

x

⁢

D

T

⁢

y

where and Σ x =Σ 0 −Σ 0 D T (σ 2 I+DΣ 0 D T ) −1 DΣ 0 ; and

estimate the super-parameter in the above equation through a maximum likelihood method or an expectation maximization method:

γ

i

=

Tr

⁡

[

B

-

1

⁡

(

Σ

x

i

+

μ

x

i

⁡

(

μ

x

i

)

T

)

]

L

,

i

=

1

,

…

⁢

,

M

,

⁢

B

=

1

M

⁢

∑

i

=

1

M

⁢

⁢

Σ

x

i

+

μ

x

i

⁡

(

μ

x

i

)

T

γ

i

,

i

=

1

,

…

⁢

,

M

,

⁢

σ

2

=

y

-

D

⁢

⁢

μ

x

2

2

+

σ

′2

⁡

[

ML

-

Tr

⁡

(

Σ

x

⁢

Σ

0

-

1

)

]

NL

,

and output final estimates of the signals after the iteration ends.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2018
From: KANG, LI; LIANG, RUNQING; HUANG, JIANJUN; HUANG, JINGXIONG
To: SHENZHEN UNIVERSITY
Reel/Frame 045527/0676 →
Continuity (2)
Continuation PCTCN2016097318 · Aug 30, 2016
Related Publication 20180198992A1 · Jul 12, 2018