IP Library Granted Patent US 11,755,639
Granted Patent B2
US 11,755,639 · App. 17/931,466 · Granted Sep 12, 2023

Systems and methods of metadata and image management for reviewing data from transmission electron microscope (TEM) sessions

Inventors: Franklin Stampley Walden, II (Raleigh, NC); John Damiano, Jr. (Holly Springs, NC); David P. Nackashi (Raleigh, NC); Daniel Stephen Gardiner (Wake Forest, NC); Mark Uebel (Morrisville, NC); Alan Philip Franks (Durham, NC); Joshua Friend (Raleigh, NC); Katherine Elizabeth Marusak (Cary, NC)
Assignee: PROTOCHIPS, INC.
G06F16/535G06F16/51G06F16/538G06F16/5866G06T1/0007G06T7/20G06T11/60G06T2200/24G06T2207/10061
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,755,639
App. No.
17/931,466
Granted
Sep 12, 2023
Kind
B2
Abstract

Disclosed herein are methods and systems of metadata management for reviewing data from microscopy experimental sessions. Image data from an experimental session is stored in an archive at one or more filepath locations, either locally or on a network. Metadata associated with the image data is stored in a database with a reference to the filepath where the raw image is stored, such that the metadata is associated in the database with the image data. A user can perform post-experimental filtering, sorting, and searching of the underlying image data using the metadata, which allows the image data to be analyzed without duplication of the image data and without manual review of each individual image. The filtered data is presented in an interactive timeline format.

Claims (35)

1. A system for generating a visual representation of experimental data from an experimental session on a transmission electron microscope using metadata from the experimental session, the system comprising:

a transmission electron microscope having a detector for capturing images of a sample during the experimental session; and

a computer system communicatively coupled to the transmission electron microscope,

the computer system comprising:

a memory; and

at least one processor configured for:

receiving, from the transmission electron microscope, a plurality of images of the sample captured by the detector of the transmission electron microscope during the experimental session;

storing, in the memory, the plurality of images of the sample received from the transmission electron microscope;

storing, in the memory, metadata associated with the plurality of images, wherein each image of the plurality of images is associated with specific metadata for said image, and wherein the specific metadata for said image includes a reference to the stored version of said image; and

generating a subset of the plurality of images, wherein the images in the subset of the plurality of images are selected based on the metadata associated with each selected image;

wherein the subset of the plurality of images is generated without storing in the memory a separate copy of each image in the subset of the plurality of images.

2. The system of claim 1 , wherein the at least one processor is further configured for generating a preview of each image in the subset of the plurality of images based on the metadata.

3. The system of claim 1 , wherein the at least one processor is further configured for displaying at least some images in the subset of the plurality of images, wherein each displayed image is displayed using a reduced size image from the stored version of each displayed image, with each displayed image being displayed in response to user input.

4. The system of claim 3 , wherein each displayed image is displayed with information related to the displayed image overlaid onto the displayed image, wherein the overlaid information is based on the metadata.

5. The system of claim 4 , wherein the at least one processor is further configured for storing the subset of the plurality of images for further review without storing a separate copy of each image in the subset of the plurality of images, wherein the subset of the plurality of images is stored using references in the metadata to the stored versions of the images in the subset of the plurality of images.

6. The system of claim 1 , wherein the at least one processor is further configured for filtering images in the plurality of images based on the metadata.

7. The system of claim 6 , wherein the images in the plurality of images are filtered to generate the subset of the plurality of images such that the subset of the plurality of images represents images from the plurality of images that have at least one similar property, wherein the at least one similar property is determined based on the metadata.

8. The system of claim 6 , wherein the filtered images reduce an active dataset of the plurality of images.

9. The system of claim 1 , wherein access to the stored versions of the images in the plurality of images occurs in approximately real-time using the metadata.

10. The system of claim 1 , wherein the images of the plurality of images are indexed based on the metadata.

11. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium storing instructions to be implemented on at least one computing device including at least one processor, the instructions when executed by the at least one processor cause the at least one computing device to perform a method of generating a visual representation of experimental data from an experimental session on a transmission electron microscope using metadata from the experimental session, the method comprising:

receiving, from the transmission electron microscope, a plurality of images of a sample captured by a detector of the transmission electron microscope during the experimental session;

storing the plurality of images of the sample received from the transmission electron microscope;

storing metadata associated with the plurality of images, wherein each image of the plurality of images is associated with specific metadata for said image, and wherein the specific metadata for said image includes a reference to the stored version of said image; and

generating a subset of the plurality of images, wherein the images in the subset of the plurality of images are selected based on the metadata associated with each selected image;

wherein the subset of the plurality of images is generated without storing a separate copy of each image in the subset of the plurality of images.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the method performed by the at least one computing device further comprises generating a preview of each image in the subset of the plurality of images based on the metadata.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the method performed by the at least one computing device further comprises displaying at least some images in the subset of the plurality of images, wherein each displayed image is displayed using a reduced size image from the stored version of each displayed image, with each displayed image being displayed in response to user input.

14. The non-transitory computer-readable storage medium of claim 13 , wherein each displayed image is displayed with information related to the displayed image overlaid onto the displayed image, wherein the overlaid information is based on the metadata.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the method performed by the at least one computing device further comprises storing the subset of the plurality of images for further review without storing a separate copy of each image in the subset of the plurality of images, wherein the subset of the plurality of images is stored using references in the metadata to the stored versions of the images in the subset of the plurality of images.

16. The non-transitory computer-readable storage medium of claim 11 , wherein the method performed by the at least one computing device further comprises filtering images in the plurality of images based on the metadata.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the images in the plurality of images are filtered to generate the subset of the plurality of images such that the subset of the plurality of images represents images from the plurality of images that have at least one similar property, wherein the at least one similar property is determined based on the metadata.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the filtered images reduce an active dataset of the plurality of images.

19. The non-transitory computer-readable storage medium of claim 11 , wherein access to the stored versions of the images in the plurality of images occurs in approximately real-time using the metadata.

20. The non-transitory computer-readable storage medium of claim 11 , wherein the images of the plurality of images are indexed based on the metadata.

Assignments (2)
SECURITY INTEREST Recorded Feb 23, 2024
From: PROTOCHIPS, INC.
To: SALEM INVESTMENT PARTNERS IV, LIMITED PARTNERSHIP
Reel/Frame 066548/0117 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2022
From: WALDEN, FRANKLIN STAMPLEY, II; DAMIANO, JOHN, JR.; NACKASHI, DAVID P.; GARDINER, DANIEL STEPHEN; UEBEL, MARK; FRANKS, ALAN PHILIP; FRIEND, JOSHUA; MARUSAK, KATHERINE ELIZABETH
To: PROTOCHIPS, INC.
Reel/Frame 061065/0830 →
Continuity (4)
Continuation 17715831 · Apr 7, 2022
Continuation PCTUS2022023892 · Apr 7, 2022
Provisional Application 63171692 · Apr 7, 2021
Related Publication 20230049691A1 · Feb 16, 2023