Machine learning techniques for tumor identification, classification, and grading
The present disclosure relates to techniques for non-invasive tumor identification, classification, and grading using mixed exam-, region-, and voxel-wise supervision. Particularly, aspects are directed to a computer implemented method that includes obtaining medical images of a subject, inputting the medical images into a three-dimensional neural network model constructed to produce a voxelwise cancer risk map of lesion occupancy and cancer grade as two output channels using an objective function having a first loss function that captures strongly supervised loss for regression in lesions and a second loss function that captures weakly supervised loss for regression in regions, generating an estimated segmentation boundary around one or more lesions, predicting a cancer grade for each pixel or voxel within the medical images, and outputting the voxelwise cancer risk map of lesion occupancy determined based on the estimated segmentation boundary and the cancer grade for each pixel or voxel within the medical images.
1 . A method for cancer detection, comprising:
obtaining medical images of a subject, the medical images include an object of interest;
inputting the medical images into a three-dimensional neural network model constructed to produce a voxelwise risk map of an object of interest occupancy and disease state grade as two output channels using an objective function comprising a first loss function and a second loss function, wherein the first loss function captures supervised loss for regression in the object of interest based on groundtruth lesion-biopsy scores and the second loss function captures supervised loss for regression in regions of the object of interest based on groundtruth region-biopsy scores;
generating, using the three-dimensional neural network model, an estimated segmentation boundary around the object of interest;
predicting, using the three-dimensional neural network model, a disease state grade for each pixel or voxel within the medical images; and
outputting, using the three-dimensional neural network, the voxelwise risk map of the object of interest occupancy determined based on the estimated segmentation boundary around the object of interest and the disease state grade for each pixel or voxel within the medical images.
2 . The method of claim 1 , wherein the object of interest is a lesion.
3 . The method of claim 1 , wherein the medical images are obtained using magnetic resonance imaging.
4 . The method of claim 1 , further comprising, prior to inputting the medical images into a three-dimensional neural network model:
inputting the medical images into a segmentation model constructed to segment a region of interest;
generating, using the segmentation model, an estimated segmentation boundary around the region of interest;
outputting, using the segmentation model, the medical images with the estimated segmentation boundary around the region of interest; and
cropping the medical images based on the estimated segmentation boundary to generate portions of the medical images comprising the region of interest,
wherein the portions of the medical images comprising the region of interest are input into the three-dimensional neural network model for producing the voxelwise risk map of the object of interest occupancy and the disease state grade as the two output channels.
5 . The method of claim 4 , wherein the region of interest is the prostate gland.
6 . The method of claim 1 , wherein:
the three-dimensional neural network model comprises a plurality of model parameters identified using a set of training data comprising a plurality of medical images with annotations associated with: (i) segmentation boundaries around objects of interest and systematic or region biopsy with disease state grades; and
the plurality of model parameters are identified using the set of training data based on minimizing the objective function.
7 . The method of claim 6 , wherein the objective function further comprises a Dice loss function, and the objective function averages the supervised loss for regression in the object of interest and the supervised loss for regression in the regions of the object of interest over each region and observed grade group.
8 . The method of claim 1 , further comprising:
determining a size, surface area, and/or volume of the one or more lesions based on the estimated segmentation boundary; and
providing: (i) the voxelwise cancer risk map, and/or (ii) a size, surface area, and/or volume of the object of interest.
9 . The method of claim 8 , further comprising: determining, by a user, a diagnosis of the subject based on (i) the voxelwise cancer risk map, and/or (ii) the size, surface area, and/or volume of the object of interest.
10 . The method of claim 9 , further comprising administering, by the user, a treatment with a compound based on (i) the voxelwise cancer risk map, (ii) the size, surface area, and/or volume of the object of interest, and/or (iii) the diagnosis of the subject.
11 . A system comprising:
one or more data processors of a local cloud server; and
a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform actions including:
obtaining medical images of a subject, the medical images include an object of interest;
inputting the medical images into a three-dimensional neural network model constructed to produce a voxelwise risk map of an object of interest occupancy and disease state grade as two output channels using an objective function comprising a first loss function and a second loss function, wherein the first loss function captures supervised loss for regression in the object of interest based on groundtruth lesion-biopsy scores and the second loss function captures supervised loss for regression in regions of the object of interest based on groundtruth region-biopsy scores;
generating, using the three-dimensional neural network model, an estimated segmentation boundary around the object of interest;
predicting, using the three-dimensional neural network model, a disease state grade for each pixel or voxel within the medical images; and
outputting, using the three-dimensional neural network, the voxelwise risk map of the object of interest occupancy determined based on the estimated segmentation boundary around the object of interest and the disease state grade for each pixel or voxel within the medical images.
12 . The system of claim 11 , wherein the object of interest is a lesion.
13 . The system of claim 11 , wherein the medical images are obtained using magnetic resonance imaging.
14 . The system of claim 11 , wherein the actions further comprise, prior to inputting the medical images into a three-dimensional neural network model:
inputting the medical images into a segmentation model constructed to segment a region of interest;
generating, using the segmentation model, an estimated segmentation boundary around the region of interest;
outputting, using the segmentation model, the medical images with the estimated segmentation boundary around the region of interest; and
cropping the medical images based on the estimated segmentation boundary to generate portions of the medical images comprising the region of interest,
wherein the portions of the medical images comprising the region of interest are input into the three-dimensional neural network model for producing the voxelwise risk map of the object of interest occupancy and the disease state grade as the two output channels.
15 . The system of claim 11 , wherein:
the three-dimensional neural network model comprises a plurality of model parameters identified using a set of training data comprising a plurality of medical images with annotations associated with: (i) segmentation boundaries around objects of interest and systematic or region biopsy with disease state grades; and
the plurality of model parameters are identified using the set of training data based on minimizing the objective function.
16 . The system of claim 15 , wherein the objective function further comprises a Dice loss function, and the objective function averages the supervised loss for regression in the object of interest and the supervised loss for regression in the regions of the object of interest over each region and observed grade group.
17 . The system of claim 11 , wherein the actions further comprise:
determining a size, surface area, and/or volume of the one or more lesions based on the estimated segmentation boundary; and
providing: (i) the voxelwise cancer risk map, and/or (ii) a size, surface area, and/or volume of the object of interest.
18 . The system of claim 17 , wherein the actions further comprise:
determining, by a user, a diagnosis of the subject based on (i) the voxelwise cancer risk map, and/or (ii) the size, surface area, and/or volume of the object of interest.
19 . The system of claim 18 , wherein the actions further comprise facilitating administration, by the user, a treatment with a compound based on (i) the voxelwise cancer risk map, (ii) the size, surface area, and/or volume of the object of interest, and/or (iii) the diagnosis of the subject.
20 . A method for cancer detection, comprising:
obtaining medical images of a subject, the medical images include an object of interest;
inputting the medical images into a three-dimensional neural network model constructed to produce a voxelwise risk map of an object of interest occupancy and disease state grade as two output channels using an objective function comprising an object loss function and one or more histogram-based loss functions, wherein the object loss function captures supervised loss for regression in the object of interest based on groundtruth lesion-biopsy scores and the histogram-based second loss function captures supervised loss for regression in regions of the object of interest based on groundtruth region-biopsy scores, and wherein each loss function of the one or more histogram-based loss functions provides a differentiable measure of accuracy in predicting object of interest occupancy, disease state grade, or properties thereof in each voxel, region, and/or exam;
generating, using the three-dimensional neural network model, an estimated segmentation boundary around the object of interest;
predicting, using the three-dimensional neural network model, a disease state grade for each pixel or voxel within the medical images; and
outputting, using the three-dimensional neural network, the voxelwise risk map of the object of interest occupancy determined based on the estimated segmentation boundary around the object of interest and the disease state grade for each pixel or voxel within the medical images.