Non-tumor segmentation to support tumor detection and analysis
The present disclosure relates machine learning techniques for segmenting non-tumor regions in specimen images to support tumor detection and analysis. Particularly, aspects of the present disclosure are directed to accessing one or more images that comprise a non-target region (e.g., a non-tumor region) and a target region (e.g., a tumor region), predicting, by a two-dimensional segmentation model, segmentation maps for the non-target region based on discriminative features encoded from the one or more images, a segmentation mask for the one or more images based on the segmentation maps, applying the segmentation mask to the one or more images to generate non-target region masked images that exclude the non-target region from the one or more images, and classifying, by an image analysis model, a biological material or structure within the target region based on a set of features extracted from the non-target region masked images.
1 . A computer-implemented method comprising:
accessing a plurality of images for a specimen, wherein one or more images of the plurality of images comprise a non-target region and a target region;
splitting the one or more images into image patches having a predetermined size;
encoding, by a two-dimensional segmentation model, the image patches into discriminative features;
projecting, by the two-dimensional segmentation model, the discriminative features onto a pixel space;
determining, by the two-dimensional segmentation model, a classification of a first biological material or structure for each pixel space based on a predetermined threshold;
predicting, by the two-dimensional segmentation model, segmentation maps for the non-target region based on the discriminative features, wherein the discriminative features are associated with the first biological material or structure;
generating a segmentation mask for the one or more images based on the segmentation maps;
applying the segmentation mask to the one or more images to generate non-target region masked images that exclude the non-target region from the one or more images;
classifying, by an image analysis model, a second biological material or structure within the target region as a type of cell or cell nuclei based on a set of features extracted from the non-target region masked images; and
providing, for the target region, the type of cell or cell nuclei classified for the second biological material or structure.
2 . The computer-implemented method of claim 1 , wherein the specimen is stained for one or more biomarkers, the first biological material or structure is a lymphoid aggregate, and the second biological material or structure is a tumor cell or cluster of tumor cells.
3 . The computer-implemented method of claim 1 , further comprising:
extracting, by the image analysis model, the set of features from the non-target region masked images; and
computing one or more metrics for the second biological material or structure based on the set of features,
wherein the providing the type of cell or cell nuclei classified for the second biological material or structure further comprises providing the one or more metrics for the second biological material or structure.
4 . The computer-implemented method of claim 1 , wherein the two-dimensional segmentation model is a modified U-Net model comprising a contracting path and an expansive path, each of the contracting path and the expansive path having a maximum of 256 channels, and one or more layers of the contracting path implement spatial drop out.
5 . The computer-implemented method of claim 4 , wherein the contracting path is configured to encode the image patches into discriminative features.
6 . The computer-implemented method of claim 5 , wherein the contracting path is configured to encode the image patches into discriminative features at multiple different levels.
7 . The computer-implemented method of claim 4 , wherein the expansive path is configured to project the discriminative features onto the pixel space.
8 . The computer-implemented method of claim 7 , wherein the expansive path is configured to project the discriminative features onto the pixel space at multiple different levels.
9 . The computer-implemented method of claim 1 , wherein the first biological material or structure is associated with the non-target region.
10 . The computer-implemented method of claim 1 , wherein the predetermined threshold is a hyperparameter that is optimized during training of the two-dimensional segmentation model.
11 . A computer-implemented method comprising:
accessing a plurality of images for a specimen, wherein one or more images of the plurality of images comprise a non-target region and a target region;
splitting the one or more images into image patches having a predetermined size;
encoding, by a two-dimensional segmentation model, the image patches into discriminative features;
projecting, by the two-dimensional segmentation model, the discriminative features onto a pixel space;
determining, by the two-dimensional segmentation model, a classification of a first biological material or structure for each pixel space based on a predetermined threshold;
predicting, by the two-dimensional segmentation model, segmentation maps for the non-target region based on the discriminative features, wherein the discriminative features are associated with the first biological material or structure;
generating a first segmentation mask for the one or more images based on the segmentation maps for the non-target region;
classifying, by an image analysis model, a second biological material or structure within the target region as a type of cell or cell nuclei based on a second set of features extracted from the one or more images;
generating a second segmentation mask for the one or more images based on the classification of the second biological material or structure within the target region;
applying the first segmentation mask and the second segmentation mask to the one or more images to generate target region and non-target region masked images that exclude the non-target region from the one or more images; and
providing the target region and non-target region masked images and the type of cell or cell nuclei classified for the second biological material or structure.
12 . The computer-implemented method of claim 11 , wherein the specimen is stained for one or more biomarkers, the first biological material or structure is a lymphoid aggregate, and the second biological material or structure is a tumor cell or cluster of tumor cells.
13 . The computer-implemented method of claim 11 , further comprising:
extracting, by the image analysis model, the second set of features from the one or more images; and
computing one or more metrics for the second biological material or structure based on the second set of features,
wherein the providing the target region and non-target region masked images and the type of cell or cell nuclei classified for the second biological material or structure further comprises providing the one or more metrics for the second biological material or structure.
14 . The computer-implemented method of claim 11 , wherein the two-dimensional segmentation model is a modified U-Net model comprising a contracting path and an expansive path, each of the contracting path and the expansive path having a maximum of 256 channels, and one or more layers of the contracting path implement spatial drop out.
15 . The computer-implemented method of claim 14 , wherein the contracting path is configured to encode the image patches into discriminative features.
16 . The computer-implemented method of claim 15 , wherein the contracting path is configured to encode the image patches into discriminative features at multiple different levels.
17 . The computer-implemented method of claim 14 , wherein the expansive path is configured to project the discriminative features onto the pixel space.
18 . The computer-implemented method of claim 17 , wherein the expansive path is configured to project the discriminative features onto the pixel space at multiple different levels.
19 . The computer-implemented method of claim 11 , wherein the first biological material or structure is associated with the non-target region.
20 . The computer-implemented method of claim 11 , wherein the predetermined threshold is a hyperparameter that is optimized during training of the two-dimensional segmentation model.