Granted Patent B2

US 12,645,996 · App. 18/162,565 · Granted Jun 2, 2026

Inventors: Hadley Fellows (Burlingame, CA); Mehrnoush Khojasteh (Redwood City, CA); Justine Larsen (Santa Clara, CA); Jim F. Martin (Mountain View, CA); Nidhin Murari (Milpitas, CA); Fahime Sheikhzadeh (Santa Clara, CA)

Assignee: VENTANA MEDICAL SYSTEMS, INC.

G06N20/00

Abstract

Claims (73)

1 . A method for active learning in a digital pathology system comprising:

accessing a machine learning model configured to use one or more parameters to process images to generate labels;

executing the machine learning model to:

for each respective digital pathology image of a plurality of digital pathology images:

generate a plurality of predicted labels for the respective digital pathology image; and

generate confidence metrics for the respective digital pathology image by generating a confidence metric for each predicted label of the plurality of predicted labels for the respective digital pathology image;

generating a plurality of certainty maps for the plurality of digital pathology images, wherein a certainty map is generated for each digital pathology image of the plurality of digital pathology images, wherein the certainty map for a given digital pathology image includes a set of values, and wherein each value of the set of values corresponds to a portion of the given digital pathology image and a predicted label of the plurality of predicted labels that is associated with the portion of the given digital pathology image;

presenting, in an interface, a subset of digital pathology images of the plurality of digital pathology images, wherein the subset of digital pathology images represents digital pathology images of the plurality of digital pathology images having confidence metrics that are below a predefined threshold, and wherein presenting the subset of digital pathology images comprises presenting a part of each digital pathology image of the subset of digital pathology images overlaid with a representation of a portion of a respective certainty map of the plurality of certainty maps;

receiving, in response to presenting the subset of digital pathology images, label input that confirms, rejects, or replaces at least one predicted label of the plurality of predicted labels; and

updating the one or more parameters of the machine learning model based on the label input.

2 . The method of claim 1 , wherein a predicted label of the plurality of predicted labels identifies a classification of a cell, an estimated quantity of a particular type of cells within at least part of at least one digital pathology image, or a combination thereof.

3 . The method of claim 1 , wherein the interface is configured to:

receive an input to represent statistics pertaining to regional confidence metrics at one magnification level;

represent confidence metrics at another magnification level; and

receive input to change the magnification level.

4 . The method of claim 1 , further comprising:

receiving a new digital pathology image;

transforming the new digital pathology image into one or more new labels that characterize the new digital pathology image or portions of the new digital pathology image; and

outputting the one or more new labels.

5 . The method of claim 1 , further comprising:

prior to generating a plurality of predicted labels for the respective digital pathology image: pre-processing an initial version of the respective digital pathology image to detect a location of each feature of interest, wherein generating the plurality of predicted labels for the respective digital pathology image includes generating a label corresponding to each detected location of interest within the respective digital pathology image.

6 . The method of claim 1 , further comprising:

selecting, for each respective predicted label of the plurality of predicted labels, a size, color, or shape of a marker based on a confidence metric for the respective predicted label, wherein the interface differentially represents predicted labels based on corresponding confidence metrics by representing each predicted label of the plurality of predicted labels using markers using a marker having the size, color, or shape.

7 . The method of claim 1 , wherein the machine learning model includes a deep neural network.

8 . A system comprising:

one or more data processors; and

a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform a set of operations including:

accessing a machine learning model configured to use one or more parameters to process images to generate labels;

executing the machine learning model to:

for each respective digital pathology image of a plurality of digital pathology images:

generate a plurality of predicted labels for the respective digital pathology image; and

generate confidence metrics for the respective digital pathology image by generating a confidence metric for each predicted label of the plurality of predicted labels for the respective digital pathology image;

generating a plurality of certainty maps for the plurality of digital pathology images, wherein a certainty map is generated for each digital pathology image of the plurality of digital pathology images, wherein the certainty map for a given digital pathology image includes a set of values, and wherein each value of the set of values corresponds to a portion of the given digital pathology image and a predicted label of the plurality of predicted labels that is associated with the portion of the given digital pathology image;

presenting, in an interface, a subset of digital pathology images of the plurality of digital pathology images, wherein the subset of digital pathology images represents digital pathology images of the plurality of digital pathology images having confidence metrics that are below a predefined threshold, and wherein presenting the subset of digital pathology images comprises presenting a part of each digital pathology image of the subset of digital pathology images overlaid with a representation of a portion of a respective certainty map of the plurality of certainty maps;

receiving, in response to presenting the subset of digital pathology images, label input that confirms, rejects, or replaces at least one predicted label of the plurality of predicted labels; and

updating the one or more parameters of the machine learning model based on the label input.

9 . The system of claim 8 , wherein a predicted label of the plurality of predicted labels identifies a classification of a cell, an estimated quantity of a particular type of cells within at least part of at least one digital pathology image, or a combination thereof.

10 . The system of claim 8 , wherein the interface is configured to:

receive an input to represent statistics pertaining to regional confidence metrics at one magnification level;

represent confidence metrics at another magnification level; and

receive input to change the magnification level.

11 . The system of claim 8 , wherein the set of operations further includes:

receiving a new digital pathology image;

transforming the new digital pathology image into one or more new labels that characterize the new digital pathology image or portions of the new digital pathology image; and

outputting the one or more new labels.

12 . The system of claim 8 , wherein the set of operations further includes:

prior to generating a plurality of predicted labels for the respective digital pathology image: pre-processing an initial version of the respective digital pathology image to detect a location of each feature of interest, wherein generating the plurality of predicted labels for the respective digital pathology image includes generating a label corresponding to each detected location of interest within the respective digital pathology image.

13 . The system of claim 8 , wherein the set of operations further includes:

selecting, for each respective predicted label of the plurality of predicted labels, a size, color, or shape of a marker based on a confidence metric for the respective predicted label, wherein the interface differentially represents predicted labels based on corresponding confidence metrics by representing each predicted label of the plurality of predicted labels using markers using a marker having the size, color, or shape.

14 . The system of claim 8 , wherein the machine learning model includes a deep neural network.

15 . A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform a set of operations including:

accessing a machine learning model configured to use one or more parameters to process images to generate labels;

executing the machine learning model to:

for each respective digital pathology image of a plurality of digital pathology images:

generate a plurality of predicted labels for the respective digital pathology image; and

generate confidence metrics for the respective digital pathology image by generating a confidence metric for each predicted label of the plurality of predicted labels for the respective digital pathology image;

generating a plurality of certainty maps for the plurality of digital pathology images, wherein a certainty map is generated for each digital pathology image of the plurality of digital pathology images, wherein the certainty map for a given digital pathology image includes a set of values, and wherein each value of the set of values corresponds to a portion of the given digital pathology image and a predicted label of the plurality of predicted labels that is associated with the portion of the given digital pathology image;

presenting, in an interface, a subset of digital pathology images of the plurality of digital pathology images, wherein the subset of digital pathology images represents digital pathology images of the plurality of digital pathology images having confidence metrics that are below a predefined threshold, and wherein presenting the subset of digital pathology images comprises presenting a part of each digital pathology image of the subset of digital pathology images overlaid with a representation of a portion of a respective certainty map of the plurality of certainty maps;

receiving, in response to presenting the subset of digital pathology images, label input that confirms, rejects, or replaces at least one predicted label of the plurality of predicted labels; and

updating the one or more parameters of the machine learning model based on the label input.

16 . The computer-program product of claim 15 , wherein a predicted label of the plurality of predicted labels identifies a classification of a cell, an estimated quantity of a particular type of cells within at least part of at least one digital pathology image, or a combination thereof.

17 . The computer-program product of claim 15 , wherein the interface is configured to:

receive an input to represent statistics pertaining to regional confidence metrics at one magnification level;

represent confidence metrics at another magnification level; and

receive input to change the magnification level.

18 . The computer-program product of claim 15 , wherein the set of operations further includes:

receiving a new digital pathology image;

transforming the new digital pathology image into one or more new labels that characterize the new digital pathology image or portions of the new digital pathology image; and

outputting the one or more new labels.

19 . The computer-program product of claim 15 , wherein the set of operations further includes:

prior to generating a plurality of predicted labels for the respective digital pathology image: pre-processing an initial version of the respective digital pathology image to detect a location of each feature of interest, wherein generating the plurality of predicted labels for the respective digital pathology image includes generating a label corresponding to each detected location of interest within the respective digital pathology image.

20 . The computer-program product of claim 15 , wherein the set of operations further includes:

selecting, for each respective predicted label of the plurality of predicted labels, a size, color, or shape of a marker based on a confidence metric for the respective predicted label, wherein the interface differentially represents predicted labels based on corresponding confidence metrics by representing each predicted label of the plurality of predicted labels using markers using a marker having the size, color, or shape.

Assignments (1)

Continuity (3)

References Cited (21)