IP Library Granted Patent US 11,056,314
Granted Patent B2
US 11,056,314 · App. 15/332,573 · Granted Jul 6, 2021

Method for acquiring intentionally limited data and the machine learning approach to reconstruct it

Inventors: Karl A. Hujsak (Evanston, IL); Vinayak P. Dravid (Evanston, IL); Benjamin D. Myers (Evanston, IL)
Assignee: Northwestern University
H01J37/222G06T5/005G06T2207/10061G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,056,314
App. No.
15/332,573
Granted
Jul 6, 2021
Kind
B2
Abstract

Aspects of the present disclosure involve a data capturing and processing system that intentionally captures data and/or data sets with missing pieces of information. The data and/or datasets may include various types of data, such as one-dimensional signals, two-dimensional images (or other images), and/or three-dimensional structures. The captured data is processed to restore missing information into the data and/or data sets, thereby enabling simultaneous pattern recognition and image recovery.

Claims (31)

1. A system for reconstructing images comprising:

a measurement system configured to capture at least one image corresponding to a sample specimen, wherein the measurement system includes a high-speed electrostatic beam blanker that includes a deflector that blocks an electron beam, which prevents portions of the sample specimen from being imaged such that portions of the at least one image are missing due to intentional under-sampling of the sample specimen, and wherein the measurement system is configured to control the high-speed electrostatic beam blanker to intentionally under-sample the sample specimen based on a pre-defined percentage-of-pixel value and a pre-defined dwell time; and

a processing device, in operable communication with the measurement system, to automatically restore the missing portions of the at least one image, wherein the processing device is configured to:

access a learned dictionary to identify latent features of the at least one image;

identify linear combinations of the latent features identified from the learned dictionary; and

reconstruct the missing portions of the at least one image according to the one or more linear combinations.

2. The system of claim 1 , wherein the measurement system is a scanning electron microscope configured to generate the electron beam to illuminate the sample specimen to generate the at least one image.

3. The system of claim 1 , wherein intentionally under-sampling the sample specimen reduces the amount of damage to the specific sample specimen during capture of the at least one image.

4. The system of claim 1 , wherein the processing device is further configured to evaluate the resolution of the at least one image after being reconstructed.

5. The system of claim 1 , wherein the sample specimen is at least one of a one-dimensional signal, a two-dimensional image, and a three-dimensional structure.

6. The system of claim 1 , wherein the measurement system is configured to apply a random sampling pattern with unfair coin toss to the sample specimen to identify specific pixels of the sample specimen that are to be imaged to satisfy the pre-defined percentage-of-pixel value.

7. A method for reconstructing images comprising:

capturing, using a measurement system, at least one image corresponding to a sample specimen, wherein the capturing includes using a high-speed electrostatic beam blanker that includes a deflector to block an electrostatic beam, which prevents portions of the sample specimen from being imaged such that portions of the at least one image are missing due to intentional under-sampling of the sample specimen, and wherein the capturing includes controlling the high-speed electrostatic beam blanker to intentionally under-sample the sample specimen based on a pre-defined percentage-of-pixel value and a pre-defined dwell time; and

automatically restoring, using a processing device, the missing portions of the at least one image by:

accessing a learned dictionary to identify latent features of the at least one image;

identifying linear combinations of the latent features that were identified from the learned dictionary; and

reconstructing the missing portions of the at least one image according to the one or more linear combinations.

8. The method of claim 7 , wherein the measurement system is a scanning electron microscope configured to generate the electron beam to illuminate the sample specimen, thereby generating the at least one image.

9. The method of claim 7 , wherein intentionally under-sampling the sample specimen reduces the amount of damage to the specific sample specimen during the capturing of the at least one image.

10. The method of claim 7 , further comprising evaluating the resolution of the at least one image after being reconstructed.

11. The method of claim 7 , wherein the sample specimen is at least one of a one-dimensional signal, a two-dimensional image, and a three-dimensional structure.

12. A non-transitory computer readable medium encoded with instructions for reconstructing images, the instructions, executable by a processing device, comprising:

capturing, at a measurement system, at least one image corresponding to a sample specimen, wherein the capturing includes using a high-speed electrostatic beam blanker that includes a deflector to block an electrostatic beam, which prevents portions of the sample specimen from being imaged such that portions of the at least one image are missing due to intentional under-sampling of the sample specimen, and wherein the capturing includes controlling the high-speed electrostatic beam blanker to intentionally under-sample the sample specimen based on a pre-defined percentage-of-pixel value and a pre-defined dwell time; and

automatically restoring, using the processing device, the missing portions of the at least one image by:

accessing a learned dictionary to identify latent features of the at least one image;

identifying linear combinations of the latent features identified from the learned dictionary; and

reconstructing the missing portions of the at least one image according to the one or more linear combinations.

13. The non-transitory computer readable medium of claim 12 , wherein the measurement system is a scanning electron microscope configured to generate the electron beam to illuminate the sample specimen to generate the at least one image.

14. The non-transitory computer readable medium of claim 12 , wherein intentionally under-sampling the sample specimen reduces the amount of damage to the specific sample specimen during the capturing of the at least one image.

15. The non-transitory computer readable medium of claim 12 , further comprising evaluating the resolution of the at least one image after being reconstructed.

16. The non-transitory computer readable medium of claim 12 , wherein the sample specimen is at least one of a one-dimensional signal, a two-dimensional image, and a three-dimensional structure.

Assignments (3)
CONFIRMATORY LICENSE Recorded Jul 26, 2017
From: NORTHWESTERN UNIVERSITY
To: NATIONAL SCIENCE FOUNDATION
Reel/Frame 043330/0803 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2016
From: DRAVID, VINAYAK P.; MYERS, BENJAMIN; HUJSAK, KARL A.
To: NORTHWESTERN UNIVERSITY
Reel/Frame 040142/0731 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2016
From: DRAVID, VINAYAK P.; MYERS, BENJAMIN; HUJSAK, KARL A.
To: NORTHWESTERN UNIVERSITY
Reel/Frame 040142/0879 →
Continuity (3)
Provisional Application 62286721 · Jan 25, 2016
Provisional Application 62245039 · Oct 22, 2015
Related Publication 20170213355A1 · Jul 27, 2017
Cited By (2)
US 12,532,085 US 12,627,902