IP Library Granted Patent US 12,067,712
Granted Patent B2
US 12,067,712 · App. 17/178,486 · Granted Aug 20, 2024

Complex system for contextual mask generation based on quantitative imaging

Inventors: Gabriel Popescu (Champaign, IL); Mikhail Eugene Kandel (Champaign, IL); Yuchen He (Urbana, IL); Nahil A. Sobh (Urbana, IL); Matthew Benjamin Wheeler (Tolono, IL); Marcello Rubessa (Middleton, WI)
Assignee: The Board of Trustees of the University of Illinois
G06T7/0012G06T2207/10056G06T2207/10072G06T2207/20081G06T2207/20084G06T2207/30024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,067,712
App. No.
17/178,486
Granted
Aug 20, 2024
Kind
B2
Abstract

A system generates a context mask based on quantitative image data. The system obtains the quantitative image data which was captured via a quantitative imaging of a sample. The quantitative image data is compared to previous quantitative image data through application of the quantitative image data to a neural network trained using the previous quantitative image data and corresponding constructed context masks. The comparison generates the context mask for the quantitative image data. The context mask provides context for the quantitative parameter values that facilities characterization of the sample.

Claims (60)

1. A method including:

obtaining specific quantitative image data captured via a quantitative imaging technique of a sample, the specific quantitative image data including, for each pixel of the specific quantitative image data, a quantitative parameter value and a pixel value, where the quantitative parameter value for the pixel is derived, at least in part, from the pixel value for the pixel;

determining a specific context mask for the specific quantitative image data by comparing the specific quantitative image data to previous quantitative image data for a previous sample via application of the specific quantitative image data to an input of a neutral network trained using constructed context masks generated based on the previous sample and the previous quantitative image data;

applying the specific context mask to the specific quantitative image data to determine a context value for the pixel, wherein the context value includes an expected dye concentration level at the pixel;

based on the pixel and the quantitative parameter value, determining a quantitative characterization for the context value; and

referencing the quantitative characterization against a structure integrity index to determine a condition of the sample.

2. The method of claim 1 , including altering the pixel value to indicate the context value.

3. The method of claim 1 , where the constructed context masks include a dye-contrast image captured of the previous sample after exposure of the previous sample to a contrast dye.

4. The method of claim 3 where the contrast dye includes a fluorescent material.

5. The method of claim 1 where the constructed context masks include operator input context designations.

6. The method of claim 5 , where the operator input context designations indicate that portions of an image depict an instance of a particular biological structure.

7. The method of claim 6 , where the context value indicates a determination that the pixel depicts, at least in part, an instance of the particular biological structure.

8. The method of claim 1 , where:

the quantitative imaging technique includes a non-destructive imaging technique; and

constructed context masks include images captured via a biologically-destructive imaging technique.

9. The method of claim 1 , where the quantitative imaging technique includes:

quantitative phase imaging;

gradient light interference microscopy;

spatial light inference microscopy;

diffraction tomography;

Fourier transform light scattering; or

any grouping of the foregoing.

10. The method of claim 1 , where obtaining the specific quantitative image data captured via a quantitative imaging technique includes capturing the pixel value via a pixel capture array positioned at a plane of a comparative effect generated by light rays traversing an objective and a processing optic.

11. A method including:

obtaining quantitative image data captured via quantitative imaging of a sample, the quantitative image data including multiple pixels, each of the multiple pixels including a respective quantitate parameter value, wherein the quantitate parameter value is derived from a raw pixel value of each pixel;

obtaining a constructed context mask for the sample, the constructed context mask including a context value for each of the multiple pixels;

creating an input-result pair by pairing the constructed context mask as a result to an input including the quantitative image data, wherein the context value includes an expected dye concentration level at the pixel;

applying the input-result pair to a neural network to adjust interneuron weights within the neural network;

determining a quantitative characterization for the context value; and

referencing the quantitative characterization against a structure integrity index to determine a condition of the sample.

12. The method of claim 11 , where applying the input-result pair to a neural network includes determining a deviation from the constructed context mask by a simulated context mask at an output of the neural network when the quantitative image data is applied as an input to the neural network when a test set of interneuron weights are present within the neural network.

13. The method of claim 12 , where determining the deviation includes determining a loss value between the constructed context mask and the simulated context mask to quantify the deviation.

14. The method of claim 13 , where applying the input-result pair to a neural network to adjust interneuron weights within the neural network includes adjusting the interneuron weights to achieve a reduction in the loss value according to an optimization algorithm.

15. The method of claim 12 , where the neural network includes a U-net neural network to support an image transformation operation between the quantitative image data and the simulated context mask.

16. The method of claim 11 , where the constructed context mask includes a dye-contrast image captured of the sample after exposure of the sample to a contrast dye.

17. The method of claim 11 , where:

the quantitative imaging includes a non-destructive imaging technique; and

constructed context mask includes an image captured via a biologically-destructive imaging technique.

18. A biological imaging device including:

a capture subsystem including:

an objective;

a processing optic positioned relative to the objective to generate a comparative effect from a light ray captured through the objective;

a pixel capture array positioned at a plane of the comparative effect;

a processing subsystem including:

memory configured to store:

raw pixel data from the pixel capture array; and

quantitative parameter values for pixels of the raw pixel data;

a neural network trained using constructed structure masks generated based on previous quantitative parameter values and previous pixel data;

a computed structure mask for the pixels;

a structure integrity index;

a processor in data communication with memory, the processor configured to:

determine the quantitative parameter values for the pixels based on the raw pixel data and the comparative effect, wherein the quantitate parameter values are derived from raw pixel values;

via execution of the neural network, determine the computed structure mask by assigning a subset of the pixels that represent portions of a selected biological structure identical mask values within the computed structure mask;

apply the computed structure mask to the raw pixel data to determine a context value for the pixel, wherein the context value includes an expected dye concentration level at the pixel;

based on ones of the quantitative parameter values corresponding to the subset of the pixels, determine a quantitative characterization of the selected biological structure; and

reference the quantitative characterization against the structure integrity index to determine a condition of the selected biological structure.

19. The biological imaging device of claim 18 , where:

the biological imaging device includes an assistive-reproductive-technology (ART) imaging device;

the biological structure includes a structure within a sperm; and

the condition includes a predicted success rate for zygote cleavage after fertilization of an ovum by the sperm.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: POPESCU, GABRIEL
To: THE BOARD OF TRUSTEES OF THE UNIVERSITY OF ILLINOIS
Reel/Frame 067959/0830 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: SOBH, NAHIL A.; KANDEL, MIKHAIL EUGENE; HE, YUCHEN; WHEELER, MATTHEW B.; RUBESSA, MARCELLO
To: THE BOARD OF TRUSTEES OF THE UNIVERSITY OF ILLINOIS
Reel/Frame 067959/0936 →
CONFIRMATORY LICENSE Recorded Nov 2, 2023
From: UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
To: NATIONAL INSTITUTES OF HEALTH (NIH), U.S. DEPT. OF HEALTH AND HUMAN SERVICES (DHHS), U.S. GOVERNMENT
Reel/Frame 065431/0365 →
Continuity (2)
Provisional Application 62978194 · Feb 18, 2020
Related Publication 20210256692A1 · Aug 19, 2021