IP Library › Granted Patent US 12,725,282
Granted Patent B2
US 12,725,282 · App. 18/425,056 · Granted Sep 1, 2026

Systems and methods for simultaneous single particle tracking, phase retrieval and PSF reconstruction

Inventors: Steve Presse (Scottsdale, AZ); Mohamadreza Fazel (Tempe, AZ); Zeliha Kilic (Memphis, TN)
Assignee: Arizona Board of Regents on Behalf of Arizona State University
G06T7/277G01N15/1429G06T2207/10056G06T2207/10064G06T2207/20076G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,725,282
App. No.
18/425,056
Granted
Sep 1, 2026
Kind
B2
Abstract

3D particle tracking and localization provide direct means to monitor details within nano-scale environments. However, a major shortcoming of 3D techniques is the sample induced aberrations due to inhomogeneous refractive index, resulting in distortion of point spread functions (PSFs), which are an important measurement tool required for these tasks in the field. This issue is particularly important when using pre-calibrated PSFs that do not take into account the sample induced aberrations. A system incorporates a Bayesian framework for simultaneous particle tracking and PSF inference directly from a given data. The system is data efficient by taking into account existing sources of uncertainty, such as uncertainty in the shape of the PSF, which is often ignored. The system is benchmarked using a wide range of synthetic and experimental data.

Claims (49)

1 . A system, comprising:

a processor in communication with a memory, the memory including instructions executable by the processor to:

access observation data including brightness data indicative of one or more light-emitting particles captured across a plurality of frames and across a plurality of planes by an imaging device, the plurality of frames having an aberration profile observable across each frame of the plurality of frames;

sample a set of joint probability values associated with observing the observation data for values of each respective parameter of a plurality of parameters of a measurement model, the plurality of parameters including:

a particle trajectory for each respective light-emitting particle of the one or more light-emitting particles across the plurality of frames; and

a set of point spread function parameters of a point spread function of the imaging device;

and

jointly infer, based on the set of joint probability values and the observation data, a set of most probable values of the plurality of parameters; and

sample probabilities associated with a background photon count per pixel for each frame of the plurality of frames using a Metropolis-Hasting procedure at each iteration of a Markov Chain Monte Carlo procedure.

2 . The system of claim 1 , the set of point spread function parameters including:

an amplitude and a phase of a pupil function associated with the aberration profile and the point spread function of the imaging device.

3 . The system of claim 1 , the plurality of parameters further including one or more of:

a diffusion coefficient;

a particle photon emission rate; and

a background photon count per pixel.

4 . The system of claim 1 , the memory including instructions executable by the processor to:

apply the Markov Chain Monte Carlo procedure to iteratively sample probability values associated with values of each respective parameter of the measurement model over a plurality of iterations.

5 . The system of claim 1 , the memory including instructions executable by the processor to:

sample probabilities associated with an amplitude and a phase of a pupil function over the plurality of frames from respective amplitude and phase posterior probability distributions using the Metropolis-Hasting procedure at each iteration of the Markov Chain Monte Carlo procedure.

6 . The system of claim 5 , wherein the amplitude and phase posterior probability distributions are respectively obtained through application of Gaussian priors on the amplitude and phase of the pupil function.

7 . The system of claim 1 , the memory including instructions executable by the processor to:

sample probabilities associated with particle trajectories over the plurality of frames from a particle trajectory posterior probability distribution using a hit-and-run sampler at each iteration of the Markov Chain Monte Carlo procedure.

8 . The system of claim 1 , the memory including instructions executable by the processor to:

sample a probability associated with a diffusion coefficient directly from a posterior probability distribution of the measurement model at each iteration of the Markov Chain Monte Carlo procedure.

9 . The system of claim 1 , the memory including instructions executable by the processor to:

sample probabilities associated with a particle photon emission rate from a light-emitting particle of the one or more light-emitting particles for each frame of the plurality of frames using the Metropolis-Hasting procedure at each iteration of the Markov Chain Monte Carlo procedure.

10 . The system of claim 1 , where the measurement model simultaneously considers each frame of the plurality of frames.

11 . A method, comprising:

accessing observation data including brightness data indicative of one or more light-emitting particles captured across a plurality of frames and across a plurality of planes by an imaging device, the plurality of frames having an aberration profile observable across each frame of the plurality of frames;

sampling a set of joint probability values associated with observing the observation data for values of each respective parameter of a plurality of parameters of a measurement model, the plurality of parameters including:

a particle trajectory for each respective light-emitting particle of the one or more light-emitting particles across the plurality of frames;

a set of point spread function parameters of a point spread function of the imaging device; and

jointly inferring, based on the set of joint probability values and the observation data, a set of most probable values of the plurality of parameters; and

sampling probabilities associated with a background photon count per pixel for each frame of the plurality of frames using a Metropolis- Hasting procedure at each iteration of a Markov Chain Monte Carlo procedure.

12 . The method of claim 11 , the plurality of parameters further including one or more of:

a diffusion coefficient;

a particle photon emission rate; and

a background photon count per pixel.

13 . The method of claim 11 , further comprising:

applying the Markov Chain Monte Carlo procedure to iteratively sample probability values associated with values of each respective parameter of the measurement model over a plurality of iterations.

14 . The method of claim 11 , further comprising:

sampling probabilities associated with an amplitude and a phase of a pupil function over the plurality of frames from respective amplitude and phase posterior probability distributions using the Metropolis-Hasting procedure at each iteration of the Markov Chain Monte Carlo procedure.

15 . The method of claim 14 , the amplitude and phase posterior probability distributions being respectively obtained through application of Gaussian priors on the amplitude and phase of the pupil function.

16 . The method of claim 11 , further comprising:

sampling probabilities associated with particle trajectories over the plurality of frames from a particle trajectory posterior probability distribution using a hit-and-run sampler at each iteration of the Markov Chain Monte Carlo procedure.

17 . The method of claim 11 , further comprising:

sampling a probability associated with a diffusion coefficient directly from a posterior probability distribution of the measurement model at each iteration of the Markov Chain Monte Carlo procedure.

18 . The method of claim 11 , further comprising:

sampling probabilities associated with a particle photon emission rate from a light-emitting particle of the one or more light-emitting particles for each frame of the plurality of frames using the Metropolis-Hasting procedure at each iteration of the Markov Chain Monte Carlo procedure.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2024
From: PRESSE, STEVE; FAZEL, MOHAMADREZA; KILIC, ZELIHA
To: ARIZONA BOARD OF REGENTS ON BEHALF OF ARIZONA STATE UNIVERSITY
Reel/Frame 067316/0042 →
CONFIRMATORY LICENSE Recorded Feb 1, 2024
From: ARIZONA STATE UNIVERSITY-TEMPE CAMPUS
To: NATIONAL INSTITUTES OF HEALTH (NIH), U.S. DEPT. OF HEALTH AND HUMAN SERVICES (DHHS), U.S. GOVERNMENT
Reel/Frame 066407/0200 →
Continuity (2)
Provisional Application 63441673 · Jan 27, 2023
Related Publication 20240257362A1 · Aug 1, 2024
References Cited (64)
US 9538157B2 · Piestun · 2017 [cited by examiner]
US 9560338B2 · Piestun · 2017 [cited by examiner]
US 11960102B2 · Xu · 2024 [cited by examiner]
US 12210167B2 · Xu · 2025 [cited by examiner]
US 20140015935A1 · Piestun · 2014 [cited by examiner]
US 20150035946A1 · Piestun · 2015 [cited by examiner]
US 20240319097A1 · Ma · 2024 [cited by examiner]
US 20250111257A1 · Goodman · 2025 [cited by examiner]
WO WO2024159127A1 · 2024 [cited by examiner]
Petrov el al., Measurement-based estimation of global pupil function in 3D localization microscopy, ARXIV ID: 1705.09694, Diigital Object Identifier: 10.1364/OE.25.007945, pp. 1-14 (Year: 2017). [cited by examiner]
Relich II, Single Particle Tracking Analysis Techniques for Live Cell Nanoscopy, DISSERTATION , Doctor of Philosophy of Physics, University of New Mexico, pp. 1-149. (Year: 2017). [cited by examiner]
Wendy K. et al.., A Parallel Evolutionary Multiple-Try Metropolis Markov Chain Monte Carlo Algorith for Sampling Spatial Partitions, arXiv:2007.11461v1 [stat.CO] Jul. 22, 2020. pp. 1-27. (Year: 2020). [cited by examiner]
Von Diexmann, L. et al. “Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking,” Chemical Reviews, 117(11), Feb. 2, 2017, pp. 7244-7275. [cited by applicant]
Fazel, M. et al., “Fluorescence Microscopy: A statistics-optics perspective.,” Reviews of Modern Physics, vol. 96, No. 2, Apr.-Jun. 2024, pp. 025003-1-025003-74. [cited by applicant]
Xu, F. et al., “Three-dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” Nature Methods, vol. 17, May 2020, pp. 531-540. [cited by applicant]
Sage, D. et al., “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nature Methods, vol. 16,May 2019, pp. 387-395. [cited by applicant]
Fazel, M. et al., “Analysis of super-resolution single molecule localization microscopy data: A tutorial,” AIP Advances 12, Jan. 3, 2022, 30 pages. [cited by applicant]
Shechtman, Y. et al., Precise Three-Dimensional Scan-Free Multiple-Particle Tracking Over Large Axial Ranges with Tetrapod Point Spread Functions. Nano Letters, May 5, 2015, pp. 4194-4199. [cited by applicant]
Manzo, C. et al., “A review of progress in single particle tracking: from methods to biophysical insights,” Reports on Progress in Physics 78, Oct. 28, 2015, pp. 1-29. [cited by applicant]
Shen, H. et al., “Single Particle Tracking: from Theory to Biophysical Applications,” Chemical Reviews 117, May 18, 2017, pp. 7331-7376. [cited by applicant]
Xu, L. et al., BNP-Track: A framework for multi-particle superresolved tracking, Apr. 15, 2023, pp. 1-30. [cited by applicant]
Sgouralis, I. et al, “Dynamic superresolution by Bayesian nonparametric image processing,” Biorxiv: the Preprint Server for Biology, Apr. 5, 2023, pp. 1-16. [cited by applicant]
Schwertner, M. et al., “Characterizing specimen induced aberrations for high NA adaptive optical microscopy,” Optics Express, vol. 12, No. 26, Dec. 27, 2004, pp. 6540-6552. [cited by applicant]
Ji, N. (2017). “Adaptive optical fluorescence microscopy,” Nature Methods, vol. 14, No. 4, Apr. 2017, pp. 374-380. [cited by applicant]
Liu, T. L. et al., “Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms,” Science 360, Apr. 20, 2018, pp. 1-13. [cited by applicant]
Prabhat, P. et al., “Simultaneous Imaging of Different Focal Planes in Fluorescence Microscopy for the Study of Cellular Dynamics in Three Dimensions,” IEEE Transactions on Nanobioscience, vol. 3, No. 4, Dec. 2004, pp. … [cited by applicant]
Ram, S., et al., “High Accuracy 3D Quantum dot Tracking with Multifocal Plane Microscopy for the Study of Fast Intracellular Dynamics in Live Cells,” Biophysical Journal, vol. 95, Dec. 2008, pp. 6025-6043. [cited by applicant]
Tahmasbi, A., et al., “Designing the focal plane spacing for multifocal plane microscopy,” Optics Express, vol. 22, No. 14, Jun. 30, 2014, pp. 16706-16721. [cited by applicant]
Huang, B. et al., “Three-Dimensional Super-Resolution Imaging by Stochastic Optical Reconstruction Microscopy,” Science, vol. 319, Feb. 8, 2008, pp. 810-813. [cited by applicant]
Pavani, S. R. P. et al., “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proceedings of the National Academy of Sciences, vol. 106, N… [cited by applicant]
Baddeley, D. et al., “Three-Dimensional Sub-100 nm Super-Resolution Imaging of Biological Samples Using a Phase Ramp in the Objective Pupil,” Nano Research, 4, Feb. 12, 2011, pp. 589-598. [cited by applicant]
Lew, M. D. et al., “Corkscrew point spread function for far-field three-dimensional nanoscale localization of pointlike objects,” Optics Letters, vol. 36, No. 2, Jan. 15, 2011, pp. 202-204. [cited by applicant]
Prasad, S. “Rotating point spread function via pupil-phase engineering,” Optics Letters, vol. 38, No. 4, Feb. 13, 2013, pp. 585-587. [cited by applicant]
Shechtman, Y. et al., “Optimal Point Spread Function Design for 3D Imaging,” Physical Review Letters, Sep. 26, 2014, pp. 133902-1-133902-5. [cited by applicant]
Jia, S. et al., “Isotropic three-dimensional super-resolution imaging with a self-bending point spread function,” Nature Photonics, vol. 8, Apr. 2014, pp. 302-306. [cited by applicant]
Weiss, L. E. et al., “Three-dimensional localization microscopy in live flowing cells,” Nature Nanotechnology, vol. 15, Jun. 2020, pp. 500-506. [cited by applicant]
Hanser, B. M., et al., “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” Journal of Microscopy, vol. 216, Pt 1, May 26, 2004, pp. 32-48. [cited by applicant]
Liu, S. et al., “Three dimensional single molecule localization using a phase retrieved pupil function,” Optics Express, vol. 21, No. 24, Dec. 2, 2013, pp. 29462-29487. [cited by applicant]
Aristov, A. et al., “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nature Communications, 2018, pp. 1-8. [cited by applicant]
Ferdman, B. et al., “VIPR: vectorial implementation of phase retrieval for fast and accurate microscopic pixel-wise pupil estimation,” Optics Express, vol. 28, No. 7, Mar. 30, 2020, pp. 10179-10198. [cited by applicant]
Li, Y. et al., “Real-time 3D single-molecule localization using experimental point spread functions,” Nature Methods, vol. 15, No. 5, May 2018, pp. 367-369. [cited by applicant]
Smith, C. S. et al., “Fast, single-molecule localization that achieves theoretically minimum uncertainty,” Nature Methods, vol. 7, No. 5, May 2010, pp. 373-375. [cited by applicant]
Fazel, M. et al., “Bayesian multiple emitter fitting using reversible jump Markov chain Monte Carlo,” Scientific Reports, Sep. 24, 2019, 10 pages. [cited by applicant]
Rasmussen, C. E., “Gaussian Processes in Machine Learning,” In Summer school on machine learning, vol. 3176, pp. 63-71. [cited by applicant]
Bryan, J. S. et al., “Inferring effective forces for Langevin dynamics using Gaussian processes,” The Journal of Chemical Physics, 152, Mar. 25, 2020, pp. 124106-1-124106-14. [cited by applicant]
Fazel, M. et al., “High resolution fluorescence lifetime maps from minimal photon counts,” Biophysical Journal, Feb. 20, 2022, p. 141a. [cited by applicant]
Metropolis, N., “Equation of state calculations by fast computing machines,” The Journal of Chemical Physics, 21(6), Jun. 1953, pp. 1087-1092. [cited by applicant]
Hastings, W. K., “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika, vol. 57, No. 1, Apr. 1970, pp. 97-109. [cited by applicant]
Fazel, M. et al. “High-precision estimation of emitter positions using Bayesian grouping of localizations,” Nature Communications, Nov. 22, 2022, pp. 1-11. [cited by applicant]
Fazel, M. et al., “Fluorescence lifetime: Beating the IRF and interpulse window,” Biophysical Journal 122, Feb. 21, 2023, pp. 672-683. [cited by applicant]
Bryan IV, J. S. et al., “Diffraction-limited molecular cluster quantification with Bayesian nonparametrics,” Nature Computational Science, 2(2), Jul. 21, 2022, pp. 1-20. [cited by applicant]
Bélisle, C. J., et al., “Hit-and-Run Algorithms for Generating Multivariate Distributions,” Mathematics of Operations Research, vol. 18, No. 2, May 1993, pp. 255-266. [cited by applicant]
Chen, M. H. et al., “General hit-and-run Monte Carlo sampling for evaluating multidimensional integrals,” Operations Research Letters 19, Feb. 1, 1996, pp. 161-169. [cited by applicant]
Saleh, B. E., & Teich, M. C. (2019). Fundamentals of photonics. john Wiley & sons. [cited by applicant]
Deschout, H. et al., “Precisely and accurately localizing single emitters in fluorescence microscopy,” Nature Methods, vol. 11, No. 3, Mar. 2014, pp. 253-266. [cited by applicant]
Bovik, A. “The Essential Guide to Image Processing,” Academic Press, Second Edition, 14 pages. [cited by applicant]
Liu, J. S. et al., “Monte Carlo Strategies in Scientific Computing,” vol. 10, Springer, 8 pages. [cited by applicant]
Descloux, A., et al., “Combined Multi-Plane Phase Retrieval and Super-Resolution optical Fluctuation Imaging for 4D Cell Milcroscopy,” Nature Photonics, 12(3), Mar. 12, 2018, pp. 165-172. [cited by applicant]
Mojiri S. et al., “Rapid multi-plane phase-contrast microscopy reveals torsional dynamics in flagellar motion,” Biomedical Optics Express 3170, vol. 12, No. 6, Jun. 1, 2021, pp. 3169-3180. [cited by applicant]
Edelstein, A. et al., “Computer control of microscopes using μManager,” Current Protocols in Molecular Biology, Supplement 92, Oct. 17, 2010, 17 pages. [cited by applicant]
Huang, F. et al., “Video-rate nanoscopy using sCMOS camera-specific single-molecule localization algorithms,” Nature Methods vol. 10, No. 7, Jul. 2013, pp. 653-658. [cited by applicant]
Tavakoli, M. et al, “Pitching single-focus confocal data analysis one photon at a time with Bayesian nonparametrics,” Physical Review X, 10. Jan. 30, 2020, 26 pages. [cited by applicant]
Jazani, S., “Single-Focus Confocal Data Analysis with Bayesian Nonparametrics,” ProQuest, Jul. 2020, 211 pages. [cited by applicant]
Liu, S. et al., “Universal inverse modelling of point spread functions for SMLM localization and microscope characterization,” Oct. 26, 2023, 36 pages. [cited by applicant]