Hybrid and accelerated ground-truth generation for duplex arrays
Methods and systems can include: accessing a digital pathology image; generating, using a first machine-learning model, a segmented image that identifies at least: a predicted diseased region and a background region in the digital pathology image; detecting depictions of a set of cells in the digital pathology image; generating, using a second machine-learning model, a cell classification for each cell of the set of cells, wherein the cell classification is selected from a set of potential classifications that indicate which, if any, of a set of biomarkers are expressed in the cell; detecting that a subset of the set of cells are within the background region; and updating the cell classification for each cell of at least some cells in the subset to be a background classification that was not included in the set of potential classifications.
1. A computer-implemented method comprising:
accessing a digital pathology image that depicts a tissue slice stained with multiple stains, each of the multiple stains staining for a corresponding biomarker of a set of biomarkers, wherein the multiple stains include at least three stains;
generating, using a first machine-learning model, a segmented image that identifies at least:
a predicted diseased region in the digital pathology image; and
a background region in the digital pathology image, wherein the background region indicates that signals that are present within the background region are not to be assessed when analyzing signals of the set of biomarkers;
detecting depictions of a set of cells in the digital pathology image;
generating, using a second machine-learning model, a cell classification for each cell of the set of cells, wherein the cell classification is selected from a set of potential classifications that indicate which, if any, of the set of biomarkers are expressed in the cell;
detecting that a subset of the set of cells in the digital pathology image are within the background region; and
in response to detecting that the subset of the set of cells in the digital pathology image are within the background region, updating the cell classification for each cell of at least some cells in the subset to be a background classification that was not included in the set of potential classifications.
2. The computer-implemented method of claim 1 , further comprising:
generating a training data set that includes the digital pathology and that includes an updated set of cell classifications that includes the updated cell classification for each cell in the subset;
training a third machine-learning model using the training data set.
3. The computer-implemented method of claim 2 , further comprising:
detecting that each cell in another subset of cells in the digital pathology image has a cell classification that is inconsistent with a region in which the cell is depicted as being located; and
setting a confidence metric for the cell classification of each cell in the other subset to be lower than a confidence level associated with different cell classifications that were not detected as being inconsistent with the region in which the cell is depicted as being located;
wherein the third machine-learning model is trained using the confidence metrics.
4. The computer-implemented method of claim 2 , further comprising:
generating a new set of cell classifications by processing a different digital-pathology image using the third machine-learning model, wherein a new subset of the new set of cell classifications correspond to the background classification; and
generating one or more metrics corresponding to a predicted diagnosis, prognosis or treatment response using the new set of cell classifications; and
outputting the one or more metrics.
5. The computer-implemented method of claim 2 , wherein the third machine-learning model includes a U-Net architecture.
6. The computer-implemented method of claim 1 , wherein updating the cell classification for each cell of the at least some cells in the subset to the background classification includes automatically update the cell classification for each cell of all cells in the subset to the background classification.
7. The computer implemented method of claim 1 , further comprising:
configuring a graphical user interface (GUI) to present an interactive screen that:
displays at least part of the segmented image;
displays, for each of at least some of the set of cells, a representation of the cell that indicates both the cell classification and a location of the depiction of the cell in the digital pathology image; and
provides a tool configured to receive input from a user that indicates an instruction to change one or more of the cell classifications of the at least some of the set of cells;
detecting an interaction with the tool that represents an instruction to change the cell classification of a particular cell of the at least some of the set of cells; and
updating, in response to the detected interaction, the changed cell classification for the particular cell,
wherein the updated set of cell classifications includes the changed cell classification for the particular cell.
8. The computer-implemented method of claim 1 , further comprising:
detecting that each cell in another subset of cells in the digital pathology image has a cell classification that is inconsistent with a region in which the cell is depicted as being located; and
automatically changing the cell classification of each cell in the other subset.
9. The computer-implemented method of claim 1 , further comprising:
generating one or more metrics corresponding to a predicted diagnosis, prognosis or treatment response using the set of cell classifications; and
outputting the one or more metrics.
10. The computer-implemented method of claim 1 , wherein the GUI is configured such that a region in the segmented image is depicted using a color that is representative of the type of region.
11. The computer-implemented method of claim 1 , wherein, for each stain of the set of stains, a target of the stain is a nuclear target.
12. The computer-implemented method of claim 1 , wherein, for each stain of the set of stains, a target of the stain is a cell-membrane target.
13. A system comprising:
one or more data processors; and
a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform a set of operations including:
accessing a digital pathology image that depicts a tissue slice stained with multiple stains, each of the multiple stains staining for a corresponding biomarker of a set of biomarkers, wherein the multiple stains include at least three stains;
generating, using a first machine-learning model, a segmented image that identifies at least:
a predicted diseased region in the digital pathology image; and
a background region in the digital pathology image, wherein the background region indicates that signals that are present within the background region are not to be assessed when analyzing signals of the set of biomarkers;
detecting depictions of a set of cells in the digital pathology image;
generating, using a second machine-learning model, a cell classification for each cell of the set of cells, wherein the cell classification is selected from a set of potential classifications that indicate which, if any, of the set of biomarkers are expressed in the cell;
detecting that a subset of the set of cells in the digital pathology image are within the background region; and
in response to detecting that the subset of the set of cells in the digital pathology image are within the background region, updating the cell classification for each cell of at least some cells in the subset to be a background classification that was not included in the set of potential classifications.
14. The system of claim 13 , wherein the set of operations further comprises:
generating a training data set that includes the digital pathology and that includes an updated set of cell classifications that includes the updated cell classification for each cell in the subset;
training a third machine-learning model using the training data set.
15. The system of claim 13 , wherein the set of operations further comprises:
configuring a graphical user interface (GUI) to present an interactive screen that:
displays at least part of the segmented image;
displays, for each of at least some of the set of cells, a representation of the cell that indicates both the cell classification and a location of the depiction of the cell in the digital pathology image; and
provides a tool configured to receive input from a user that indicates an instruction to change one or more of the cell classifications of the at least some of the set of cells;
detecting an interaction with the tool that represents an instruction to change the cell classification of a particular cell of the at least some of the set of cells; and
updating, in response to the detected interaction, the changed cell classification for the particular cell,
wherein the updated set of cell classifications includes the changed cell classification for the particular cell.
16. The system of claim 13 , wherein the set of operations further comprises:
detecting that each cell in another subset of cells in the digital pathology image has a cell classification that is inconsistent with a region in which the cell is depicted as being located; and
automatically changing the cell classification of each cell in the other subset.
17. The system of claim 13 , wherein the set of operations further comprises:
generating one or more metrics corresponding to a predicted diagnosis, prognosis or treatment response using the set of cell classifications; and
outputting the one or more metrics.
18. The system of claim 13 , wherein the GUI is configured such that a region in the segmented image is depicted using a color that is representative of the type of region.
19. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform a set of operations comprising:
accessing a digital pathology image that depicts a tissue slice stained with multiple stains, each of the multiple stains staining for a corresponding biomarker of a set of biomarkers, wherein the multiple stains include at least three stains;
generating, using a first machine-learning model, a segmented image that identifies at least:
a predicted diseased region in the digital pathology image; and
a background region in the digital pathology image, wherein the background region indicates that signals that are present within the background region are not to be assessed when analyzing signals of the set of biomarkers;
detecting depictions of a set of cells in the digital pathology image;
generating, using a second machine-learning model, a cell classification for each cell of the set of cells, wherein the cell classification is selected from a set of potential classifications that indicate which, if any, of the set of biomarkers are expressed in the cell;
detecting that a subset of the set of cells in the digital pathology image are within the background region; and
in response to detecting that the subset of the set of cells in the digital pathology image are within the background region, updating the cell classification for each cell of at least some cells in the subset to be a background classification that was not included in the set of potential classifications.