Systems and methods for platform agnostic whole body image segmentation
Presented herein are systems and methods that provide for automated analysis of three-dimensional (3D) medical images of a subject in order to automatically identify specific 3D volumes within the 3D images that correspond to specific anatomical regions (e.g., organs and/or tissue). Notably, the image analysis approaches described herein are not limited to a single particular organ or portion of the body. Instead, they are robust and widely applicable, providing for consistent, efficient, and accurate detection of anatomical regions, including soft tissue organs, in the entire body. In certain embodiments, the accurate identification of one or more such volumes is used to automatically determine quantitative metrics that represent uptake of radiopharmaceuticals in particular organs and/or tissue regions. These uptake metrics can be used to assess disease state in a subject, determine a prognosis for a subject, and/or determine efficacy of a treatment modality.
1. A method for automatically processing 3D images to identify, and measure uptake of radiopharmaceutical in, cancerous lesions within a subject having or at risk for a cancer, the method comprising:
(a) receiving, by a processor of a computing device, a 3D anatomical image of a subject obtained using an anatomical imaging modality, wherein the 3D anatomical image comprises a graphical representation of tissue within the subject;
(b) automatically identifying, by the processor, using one or more machine learning modules, within the 3D anatomical image:
a first skeletal volume comprising a graphical representation of one or more bones of the subject;
a first aorta volume comprising a graphical representation of at least a portion of an aorta of the subject; and
a first liver volume comprising a graphical representation of a liver of the subject;
(c) determining, by the processor, a 3D segmentation map representing a plurality of 3D segmentation masks, including a skeletal mask representing the identified first skeletal volume, an aorta mask representing the identified first aorta volume, and a liver mask representing the identified first liver volume;
(d) receiving, by the processor, a 3D functional image of the subject obtained using a functional imaging modality;
(e) automatically identifying, within the 3D functional image, using the 3D segmentation map:
a second skeletal volume corresponding to the first identified skeletal volume, within the 3D anatomical image;
a second aorta volume corresponding to the first aorta volume, identified within the 3D anatomical image; and
a second liver volume corresponding to the first liver volume, identified within the 3D anatomical image;
(f) automatically detecting, by the processor, within the second skeletal volume, one or more hotspots determined to represent lesions based on intensities of voxels within the second skeletal volume; and
(g) determining, by the processor, for each of the one or more detected hotspots, an individual hotspot index value by:
determining an aorta reference intensity level based on a measure of intensity of voxels within the second aorta volume;
determining a liver reference intensity level based on a measure of intensity of voxels within the second liver volume; and
for each individual detected hotspot:
determining a corresponding individual hotspot intensity level based on a measure of intensity of voxels of the detected hotspot; and
determining a corresponding individual hotspot index level from the individual hotspot intensity level, the aorta reference intensity level, and the liver reference intensity level.
2. The method of claim 1 , comprising determining, by the processor, an overall index value indicative of a cancer status of the subject based on the individual hotspot index values of at least a portion of the one or more detected hotspots.
3. The method of claim 1 , wherein the subject has or is at risk for prostate cancer.
4. The method of claim 1 , wherein:
step (b) comprises automatically identifying, within the 3D anatomical image, a first prostate volume comprising a graphical representation of a prostate of the subject,
the 3D segmentation map determined at step (c) further includes a prostate mask representing the identified first prostate volume,
step (e) comprises automatically identifying, within the 3D functional image, a second prostate volume corresponding to the first identified prostate volume, within the 3D anatomical image,
step (f) comprises automatically detecting one or more hotspots in the second prostate volume, and the method further comprises:
determining, by the processor, (i) an overall bone index value indicative of a lesion content in the one or more bones of the subject based on the individual hotspot index values of at least a portion of the one or more detected hotspots located in the second skeletal volume and (ii) an overall prostate index value indicative of a lesion content in the prostate of the subject based on the individual hotspot index values of at least a portion of the one or more detected hotspots located in the second prostate volume.
5. The method of claim 1 , wherein the subject has or is at risk for metastatic cancer.
6. A system for automatically processing 3D images to identify, and measure uptake of radiopharmaceutical in, cancerous lesions within a subject having or at risk for a cancer, the system comprising:
a processor of a computing device; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
(a) receive a 3D anatomical image of a subject obtained using an anatomical imaging modality, wherein the 3D anatomical image comprises a graphical representation of tissue within the subject;
(b) automatically identify using one or more machine learning modules, within the 3D anatomical image:
a first skeletal volume comprising a graphical representation of one or more bones of the subject;
a first aorta volume comprising a graphical representation of at least a portion of an aorta of the subject; and
a first liver volume comprising a graphical representation of a liver of the subject;
(c) determine a 3D segmentation map representing a plurality of 3D segmentation masks, including a skeletal mask representing the identified first skeletal volume, an aorta mask representing the identified first aorta volume, and a liver mask representing the identified first liver volume;
(d) receive a 3D functional image of the subject obtained using a functional imaging modality;
(e) automatically identify, within the 3D functional image, using the 3D segmentation map:
a second skeletal volume corresponding to the first identified skeletal volume, within the 3D anatomical image;
a second aorta volume corresponding to the first aorta volume, identified within the 3D anatomical image; and
a second liver volume corresponding to the first liver volume, identified within the 3D anatomical image;
(f) automatically detect, within the second skeletal volume, one or more hotspots determined to represent lesions based on intensities of voxels within the second skeletal volume; and
(g) determine, for each of the one or more detected hotspots, an individual hotspot index value by:
determining an aorta reference intensity level based on a measure of intensity of voxels within the second aorta volume;
determining a liver reference intensity level based on a measure of intensity of voxels within the second liver volume; and
for each individual detected hotspot:
determining a corresponding individual hotspot intensity level based on a measure of intensity of voxels of the detected hotspot; and
determining a corresponding individual hotspot index level from the individual hotspot intensity level, the aorta reference intensity level, and the liver reference intensity level.
7. The system of claim 6 , wherein the instructions cause the processor to determine an overall index value indicative of a cancer status of the subject based on the individual hotspot index values of at least a portion of the one or more detected hotspots.
8. The system of claim 6 , wherein the subject has or is at risk for prostate cancer.
9. The system of claim 6 , wherein the instructions cause the processor to:
at step (b), automatically identify, within the 3D anatomical image, a first prostate volume comprising a graphical representation of a prostate of the subject;
at step (c), include a prostate mask representing the identified first prostate volume in the determined 3D segmentation map;
at step (e), automatically identify, within the 3D functional image, a second prostate volume corresponding to the first identified prostate volume, within the 3D anatomical image,
at step (f), automatically detect one or more hotspots in the second prostate volume; and
determine (i) an overall bone index value indicative of a lesion content in the one or more bones of the subject based on the individual hotspot index values of at least a portion of the one or more detected hotspots located in the second skeletal volume and (ii) an overall prostate index value indicative of a lesion content in the prostate of the subject based on the individual hotspot index values of at least a portion of the one or more detected hotspots located in the second prostate volume.
10. The system of claim 6 , wherein the subject has or is at risk for metastatic cancer.