IP Library Granted Patent US 12,430,821
Granted Patent B2
US 12,430,821 · App. 18/250,650 · Granted Sep 30, 2025

Method and a system for compressed ultrafast tomographic imaging

Inventors: Jinyang Liang (Boucherville, CA); Yingming Lai (Longueuil, CA)
Assignee: INSTITUT NATIONAL DE LA RECHERCHE SCIENTIFIQUE
G06T11/003G06T9/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,430,821
App. No.
18/250,650
Granted
Sep 30, 2025
Kind
B2
Abstract

A method and a system for imaging a transient event, the method comprising passively recording spatiotemporal projections of the transient event in an angular range from −45° to +45°; and processing the recorded spatiotemporal projections by compressed sensing tomographic image reconstruction to recover the transient event. The system comprises an imaging unit and a shearing unit for imaging a dynamic event to different positions, a detector that records data by spatially integrating over each pixel and temporally integrating; and a processing unit that reconstructs the dynamic event from said data by compressed sensing tomographic image reconstruction.

Claims (191)

1. A method for compressed ultrafast tomographic imaging of dynamic events, comprising data acquisition and image reconstruction from acquired data; wherein said data acquisition comprises acquiring spatiotemporal projections I(x,y,t) of a dynamic event, temporally shearing the acquired projections and spatiotemporally integrating resulting sheared projections, wherein the dynamic event is directly imaged by a streak camera with shearing velocities v i (i=1, 2, . . . , N) where N is an integer; a 2D readout camera spatially integrates over each pixel and temporally integrates over the exposure time to a snapshot E i , and the spatiotemporal projections of I(x,y,t) are r described in the x-y-t domain as follows:

E=TS I ( x,y,t ),

where E=[E 1 , E 2 , E N ] T and S=[S 1 , S 2 , S N ] T′ , an angle of a i th spatiotemporal projection being

θ

i

=

tan

-

1

(

v

i

"\[LeftBracketingBar]"

v

max

"\[RightBracketingBar]"

)

,

where |v max | is a maximum shearing velocity of the streak camera; and

wherein said image reconstruction comprises building a sparse matrix and recovering the dynamic event by solving an optimization problem defined with an initialization Î 0 =(TS) T E, as follows:

I

^

=

arg

min

I

{

1

2

E

-

TSI

2

2

+

τ

Φ

TV

(

I

)

}

,

where τ is a regularization parameter, and Φ TV (l) is a regularization function of total variation TV; and resulting reconstructed spatiotemporal projections I(x,y,t) of the dynamic event have a sequence depth of N t =rT s , where T s is a sweep time,

r

=

"\[LeftBracketingBar]"

v

max

"\[RightBracketingBar]"

P

c

is an imaging speed, and P c is a pixel size of the readout camera, each frame in the spatiotemporal projections I(x,y,t) having a (x,y) size of N x ≤N b and N y ≤N v −N t +1 pixels N b and N v being horizontal and vertical pixel counts of the readout camera, the reconstruction being done by a processing unit.

2. A method for imaging a transient event, comprising passively recording spatiotemporal projections of the transient event in an angular range from −45° to +45°; and processing the recorded spatiotemporal projections by compressed sensing tomographic image reconstruction to recover the transient event, the method comprising data acquisition and image reconstruction from acquired data; wherein said data acquisition comprises acquiring spatiotemporal projections I(x,y,t) of the transient event, temporally shearing the acquired projections and spatiotemporally integrating resulting sheared projections, wherein the transient event is directly imaged by a streak camera with shearing velocities v i (i=1, 2, N) where N is an integer; a 2D readout camera spatially integrates over each pixel and temporally integrates over the exposure time to a snapshot E i , and the spatiotemporal projections of I(x,y,t) are r described in the x-y-t domain as follows:

E=TS I ( x,y,t ),

where E=[E 1 , E 2 , E N ] T and S=[S 1 , S 2 , S N ] T′ , an angle of a i th spatiotemporal projection being

θ

i

=

tan

-

1

(

v

i

"\[LeftBracketingBar]"

v

max

"\[RightBracketingBar]"

)

,

where |v max | is a maximum shearing velocity of the streak camera; and

wherein said image reconstruction comprises building a sparse matrix and recovering the dynamic event by solving an optimization problem defined with an initialization Î 0 =(TS) T E, as follows:

I

^

=

arg

min

I

{

1

2

E

-

TSI

2

2

+

τ

Φ

TV

(

I

)

}

,

where τ is a regularization parameter, and Φ TV (l) is a regularization function of total variation TV; and resulting reconstructed spatiotemporal projections I(x,y,t) of the dynamic event have a sequence depth of N t =rT s , where T s is a sweep time,

r

=

"\[LeftBracketingBar]"

v

max

"\[RightBracketingBar]"

P

c

is an imaging speed, and P c is a pixel size of the readout camera, each frame in the spatiotemporal projections I(x,y,t) having a (x,y) size of N x ≤N b and N y ≤N v −N t +1 pixels N b and N v being horizontal and vertical pixel counts of the readout camera, the reconstruction being done by a processing unit.

3. The method of claim 2 , comprising using an image-converter streak camera.

4. The method of claim 2 , comprising using a rotating-mirror streak camera.

5. The method of claim 2 , comprising acquiring 2D streak images with different shear velocities; treating the streak images at the different shear velocities as corresponding different angular projections in temporal dimension; and processing the streak images by the compressed sensing tomographic image reconstruction to recover the transient event.

6. The method of claim 2 , comprising temporal shearing and spatiotemporal integration.

7. A system for compressed ultrafast tomographic imaging, comprising an imaging unit, a shearing unit, a detector, and a processing unit; wherein said imaging unit and said shearing unit image a dynamic event to different positions; said detector records data by spatially integrating over each pixel and temporally integrating; and said processing unit reconstruct the dynamic event from said data by compressed sensing tomographic image reconstruction, the system comprising a streak camera imaging the dynamic event by taking N images with shearing velocities v i (i=1, 2, N) where N is an integer; and a 2D readout camera that spatially integrates over each pixel and temporally integrates over the exposure time to a snapshot E i , yielding spatiotemporal projections of I(x,y,t) described in a x-y-t domain as follows:

E=TS I ( x,y,t ),

where E=[E 1 , E 2 , E N ] T and S=[S 1 , S 2 , S N ] T′ , an angle of a i th spatiotemporal projection being

θ

i

=

tan

-

1

(

v

i

"\[LeftBracketingBar]"

v

max

"\[RightBracketingBar]"

)

,

where |v max | is a maximum shearing velocity of the streak camera; wherein said processing image reconstruct the dynamic event by building a sparse matrix and recovering the dynamic event by solving an optimization problem defined with an initialization Î 0 =(TS) T E, as follows:

I

^

=

arg

min

I

{

1

2

E

-

TSI

2

2

+

τ

Φ

TV

(

I

)

}

,

where τ is a regularization parameter, and Φ TV (l) is a regularization function of total variation TV; and resulting reconstructed spatiotemporal projections I(x,y,t) of the dynamic event have a sequence depth of N t =rT s , where T s is a sweep time,

r

=

"\[LeftBracketingBar]"

v

max

"\[RightBracketingBar]"

P

c

is an imaging speed, and P c is a pixel size of the readout camera, each frame in the spatiotemporal projections I(x,y,t) having a (x,y) size of N x ≤N b and N y ≤N v −N t +1 pixels N b and N v being horizontal and vertical pixel counts of the readout camera.

8. The system of claim 7 , wherein said imaging unit and said shearing unit are provided as a streak camera.

9. The system of claim 7 , wherein said imaging unit and said shearing unit are provided as an image-converter streak camera.

10. The system of claim 7 , wherein said imaging unit and said shearing unit are provided as a rotating-mirror streak camera.

11. The system of claim 7 , wherein said shearing unit comprises a galvanometer scanner.

12. The system of claim 7 , wherein said detector is one of: electron-multiplying charge-coupled devices (CCD), complementary metal-oxide-semiconductor (CMOS) cameras, and scientific CMOS cameras.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2023
From: LIANG, JINYANG
To: INSTITUT NATIONAL DE LA RECHERCHE SCIENTIFIQUE
Reel/Frame 064026/0351 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2023
From: LAI, YINGMING
To: INSTITUT NATIONAL DE LA RECHERCHE SCIENTIFIQUE
Reel/Frame 064026/0393 →
Continuity (2)
Provisional Application 63109025 · Nov 3, 2020
Related Publication 20230401765A1 · Dec 14, 2023
References Cited (63)
US 7463362B2 · Lasker et al. · 2008 [cited by applicant]
US 9014451B2 · Beg et al. · 2015 [cited by applicant]
US 9492089B2 · Hielscher et al. · 2016 [cited by applicant]
US 9506740B2 · Brezinzki et al. · 2016 [cited by applicant]
US 9775511B2 · Kumar et al. · 2017 [cited by applicant]
US 9835840B2 · Jalali et al. · 2017 [cited by applicant]
US 9867542B2 · Wu et al. · 2018 [cited by applicant]
US 10458921B2 · Brenner et al. · 2019 [cited by applicant]
US 10502544B2 · Liu et al. · 2019 [cited by applicant]
US 20040030255A1 · Alfano · 2004 [cited by examiner]
US 20170105618A1 · Schmoll et al. · 2017 [cited by applicant]
US 20180224552A1 · Wang et al. · 2018 [cited by applicant]
US 20230125131A1 · Gao · 2023 [cited by examiner]
CN 110160440 · 2019 [cited by applicant]
CN 210036591 · 2020 [cited by applicant]
JP 2015523578 · 2015 [cited by applicant]
JP 2018510962 · 2019 [cited by applicant]
KR 101263326 · 2013 [cited by applicant]
TW 201333448 · 2013 [cited by applicant]
TW 2015523578 · 2014 [cited by applicant]
International Search Report & Written Opinion issued in parent PCT application No. PCT/CA2021/051516 dated Jan. 24, 2022. [cited by applicant]
CN210036591U—Machine Translation_English. [cited by applicant]
CN110160440A—Machine Translation_English. [cited by applicant]
JP2019510962A—Machine Translation_English. [cited by applicant]
JP2015523578A—Machine Translation_English. [cited by applicant]
KR101266326B1—Machine Translation_English. [cited by applicant]
TW201333448A—Machine Translation_English. [cited by applicant]
TWM488016U—Machine Translation_English. [cited by applicant]
J. Liang and L. V. Wang, “Single-shot ultrafast optical imaging,” Optica 5, 1113-1127 (2018). [cited by applicant]
H. Mikami, L. Gao, and K. Goda, “Ultrafast optical imaging technology: principles and applications of emerging methods,” Nanophotonics 5, 497-509 (2016). [cited by applicant]
A. Velten et al., “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Communications 3, 745 (2012). [cited by applicant]
A. Velten et al.,, “Videos of light filamentation in air,” Journal of Physics B: Atomic, Molecular and Optical Physics 48, 094020 (2015). [cited by applicant]
G. Satat, et al., “All Photons Imaging Through Volumetric Scattering,” Scientific Reports 6, 33946 (2016). [cited by applicant]
R. Kienberger et al., “Atomic transient recorder,” Nature 427, 817-821 (2004). [cited by applicant]
J. A. Russell et al., “Characterization of Fluorescence Lifetime of Photofrin and Delta-Aminolevulinic Acid Induced Protoporphyrin IX in Living Cells Using Single- and Two-Photon Excitation,” IEEE Journal of Selected To… [cited by applicant]
Y. Ping et al., “Enhanced energy coupling for indirectly driven inertial confinement fusion,” Nature Physics 15, 138-141 (2019). [cited by applicant]
Q. Hu et al., “Ultrafast fluorescent decay induced by metal-mediated dipole-dipole interaction in two-dimensional molecular aggregates,” Proceedings of the National Academy of Sciences 114, 10017 (2017). [cited by applicant]
Hamamatsu Photonics, “Guide to Streak Cameras”, retrieved https://www.hamamatsu.com/resources/pdf/sys/SHSS0006E_STREAK.pdf.(2008). [cited by applicant]
Axis Photonique Incorporation, “Axis-PX: Subpicosecond X-Ray Streak Camera”, retrieved http://www.axis-photon.com/streak-camera/axis-px-subpicosecond-x-ray-streak-camera/.(2020). [cited by applicant]
D. Faccio et al., “A trillion frames per second: the techniques and applications of light-in-flight photography,” Reports on Progress in Physics 81, 105901 (2018). [cited by applicant]
M. Drabbels et al., “Demonstration of a far-infrared streak camera,” IEEE Journal of Quantum Electronics 34, 2138-2144 (1998). [cited by applicant]
J. B. M. Warntjes et al., “Atomic streak camera operating in the extreme ultraviolet,” Review of Scientific Instruments 72, 3205-3207 (2001). [cited by applicant]
C. H. Sarantos et al., “Solid-state ultrafast all-optical streak camera enabling high-dynamic-range picosecond recording,” Optics Letters 35, 1389-1391 (2010). [cited by applicant]
J. Itatani et al., “Attosecond Streak Camera,” Physical Review Letters 88, 173903 (2002). [cited by applicant]
A. Velten et al., “Femto-photography: capturing and visualizing the propagation of light,” Acm Transactions on Graphics 32, 8 (2013). [cited by applicant]
L. Gao et al., “Single-shot compressed ultrafast photography at one hundred billion frames per second,” Nature 516, 74-77 (2014). [cited by applicant]
A. Tsikouras et al., “High-speed multifocal array scanning using refractive window tilting,” Biomedical Optics Express 6, 3737-3747 (2015). [cited by applicant]
B. Heshmat et al., “Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array,” in CLEO: 2014, OSA Technical Digest (online) (Optical Society of America, 2014). [cited by applicant]
R. H. H. Scott et al., “A study of fast electron energy transport in relativistically intense laser-plasma interactions with large density scalelengths,” Physics of Plasmas 19, 053104 (2012). [cited by applicant]
J. Liang et al., “Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution,” Nature Communications 11, 5252 (2020). [cited by applicant]
M. Nakatsutsumi et al., “Space and time resolved measurements of the heating of solids to ten million kelvin by a petawatt laser,” New Journal of Physics 10, 043046 (2008). [cited by applicant]
Y. Lai et al., “Single-Shot Ultraviolet Compressed Ultrafast Photography,” Laser & Photonics Reviews 14, 2000122 (2020). [cited by applicant]
Z. Li et al., “Single-shot tomographic movies of evolving light-velocity objects,” Nature Communications 5, 3085 (2014). [cited by applicant]
N. H. Matlis et al., “Single-shot ultrafast tomographic imaging by spectral multiplexing,” Nature Communications 3, 1111 (2012). [cited by applicant]
E. Tokunaga et al., “Frequency-domain interferometer for femtosecond time-resolved phase spectroscopy,” Optics Letters 17, 1131-1133 (1992). [cited by applicant]
J. Liang, “Punching holes in light: Recent progress in single-shot coded-aperture optical imaging,” Reports on Progress in Physics (2020). [cited by applicant]
K. Egiazarian et al., “Compressed Sensing Image Reconstruction via Recursive Spatially Adaptive Filtering,” in 2007 IEEE International Conference on Image Processing, 2007), I-549-I-552. [cited by applicant]
C. C. Paige et al., “LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares,” ACM Transactions on Mathematical Software (TOMS) 8, 43-71 (1982). [cited by applicant]
J. M. Bioucas-Dias et al., “A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration,” IEEE Transactions on Image Processing 16, 2992-3004 (2007). [cited by applicant]
https://link.springer.com/book/10.1007/978-3-642-00296-0—J. Benesty et al., “Pearson Correlation Coefficient,” in Noise Reduction in Speech Processing, I. Cohen, Y. Huang, J. Chen, and J. Benesty, eds. (Springer Berlin … [cited by applicant]
X. Liu, et al., “Single-shot compressed optical-streaking ultra-high-speed photography,” Optics Letters 44, 1387-1390 (2019). [cited by applicant]
G. D. Gautam et al., “Pulsed Nd:YAG laser beam drilling: A review,” Optics & Laser Technology 100, 183-215 (2018). [cited by applicant]
V. Ntziachristos et al., “Fluorescence molecular tomography resolves protease activity in vivo,” Nature Medicine 8, 757-761 (2002). [cited by applicant]