Systems and methods for automated and interactive analysis of bone scan images for detection of metastases
Presented herein are systems and methods that provide for improved computer aided display and analysis of nuclear medicine images. In particular, in certain embodiments, the systems and methods described herein provide improvements to several image processing steps used for automated analysis of bone scan images for assessing cancer status of a patient. For example, improved approaches for image segmentation, hotspot detection, automated classification of hotspots as representing metastases, and computation of risk indices such as bone scan index (BSI) values are provided.
1. A method for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the method comprising:
(a) accessing, by a processor of a computing device, a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segmenting, by the processor, each image in the bone scan image set to identify one or more skeletal regions of interest, each corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images, wherein the one or more skeletal regions of interest comprise at least one of (i) and (ii):
(i) a femur region corresponding to a portion of a femur of the human subject, said femur portion encompassing at least three quarters of the femur along its length; and
(ii) a humerus region corresponding to a portion of a humerus of the human subject, said humerus portion encompassing at least three quarters of the humerus along its length;
(c) automatically detecting, by the processor, an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images, said automatically detecting comprising identifying the one or more hotspots using intensities of pixels in the annotated set of images and using one or more region-dependent threshold values, and wherein the one or more region dependent threshold values include one or more values associated with the femur region and/or the humerus region that provide enhanced hotspot detection sensitivity in the femur region and/or the humerus region to compensate for reduced uptake of the agent therein;
(d) for each hotspot in the initial set of hotspots, extracting, by the processor, a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculating, by the processor, a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot; and
(f) causing, by the processor, rendering of a graphical representation of at least a portion of the initial set of hotspots for display within a graphical user interface (GUI),
wherein step (b) comprises:
comparing each member of the bone scan image set with a corresponding atlas image of an atlas image set, each atlas image comprising one or more identifications of the one or more skeletal regions of interest, said skeletal regions of interest including (i) the femur region, comprising at least a portion of a knee region of the human subject, and/or (ii) the humerus region, comprising at least a portion of an elbow region of the human subject; and
for each image of the bone scan image set, registering the corresponding atlas image with the image of the bone scan image set using the identified knee region and/or the identified elbow region in the image as (a) landmark(s), such that the identifications of the one or more skeletal regions of interest of the atlas image are applied to the image of the bone scan image set.
2. The method of claim 1 , wherein a location of at least one detected hotspot of the initial hotspot set corresponds to a physical location in or on a femur more than three quarters of a distance along the femur from an end of the femur oriented toward a hip of the human subject to an end of the femur oriented toward a knee of the human subject.
3. The method of claim 1 , wherein a location of at least one detected hotspot of the initial hotspot set corresponds to a physical location in or on a humerus more than three quarters of a distance along the humerus from an end of the humerus oriented toward a shoulder of the human subject to an end of the humerus oriented toward an elbow of the human subject.
4. The method of claim 1 , where step (c) comprises:
identifying, by the processor, healthy tissue regions in the images of the bone scan image set determined not to include any hotspots;
calculating, by the processor, a normalization factor such that a product of the normalization factor and an average intensity of the identified healthy tissue regions is a pre-defined intensity level; and
normalizing, by the processor, the images of the bone scan image set by the normalization factor.
5. The method of claim 1 , further comprising:
(g) calculating, by the processor, one or more risk index values for the human subject based at least in part on a computed fraction of the skeleton of the human subject occupied by the initial set of hotspots.
6. The method of claim 5 , wherein at least one of the risk index values is indicative of a risk of the human subject having and/or developing metastatic cancer.
7. The method of claim 6 , wherein the metastatic cancer is metastatic prostate cancer.
8. The method of claim 5 , wherein at least one of the risk index values is indicative of the human subject having a particular state of metastatic cancer.
9. The method of claim 1 , comprising:
(h) selecting, by the processor, a first subset of the initial set of hotspots based at least in part on the metastasis likelihood values; and
(i) causing, by the processor, rendering of a graphical representation of the first subset for display within a graphical user interface (GUI).
10. The method of claim 9 , further comprising:
(j) calculating, by the processor, one or more risk index values for the human subject based at least in part on a computed fraction of the skeleton of the human subject occupied by the first subset of hotspots.
11. The method of claim 1 , comprising:
(k) receiving, by the processor, via the GUI, a user selection of a second subset of the initial set of hotspots; and
(l) calculating, by the processor, one or more risk index values for the human subject based at least in part on a computed fraction of the skeleton of the human subject occupied by the second subset of hotspots.
12. The method of claim 1 , wherein the processor is a processor of a cloud-based system.
13. The method claim 1 , wherein the GUI is part of a general Picture Archiving and Communications System (PACS).
14. The method of claim 1 , wherein the agent comprises technetium 99m methylenediphosphonate ( 99m Tc-MDP).
15. A method for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the method comprising:
(a) accessing, by a processor of a computing device, a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segmenting, by the processor, each image in the bone scan image set to identify one or more skeletal regions of interest, each skeletal region of interest corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images;
(c) automatically detecting, by the processor, an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images, said automatically detecting comprising:
using (i) intensities of pixels in the annotated set of images and (ii) a plurality of preliminary threshold values to detect a set of potential hotspots;
computing a global threshold scaling factor using the set of potential hotspots;
adjusting the plurality of preliminary threshold values using the global threshold scaling factor, thereby obtaining a plurality of adjusted threshold values; and
using (i) intensities of pixels in the annotated set of images and (ii) the plurality of adjusted threshold values to identify the initial set of hotspots;
(d) for each hotspot in the initial set of hotspots, extracting, by the processor, a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculating, by the processor, a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot; and
(f) causing, by the processor, rendering of a graphical representation of at least a portion of the initial set of hotspots for display within a graphical user interface (GUI).
16. The method of claim 15 , wherein the global threshold scaling factor is a function of a measure of disease burden for the human subject, and wherein the adjusting the plurality of preliminary threshold values performed at step (c) comprises decreasing the adjusted threshold values as disease burden increases so as to compensate for an underestimation of hotspot area that occurs with increasing disease burden.
17. The method of claim 15 , wherein the global threshold scaling factor is a function of a fraction of the identified skeletal regions occupied by the set of potential hotspot set.
18. The method of claim 15 , wherein the global threshold scaling factor is based on a risk index value calculated using the set of potential hotspots.
19. A method for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the method comprising:
(a) accessing, by a processor of a computing device, a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segmenting, by the processor, each image in the bone scan image set to identify one or more skeletal regions of interest, each skeletal region of interest corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images;
(c) automatically detecting, by the processor, an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images;
(d) for each hotspot in the initial set of hotspots, extracting, by the processor, a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculating, by the processor, a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot;
(f) selecting, by the processor, a first subset of the initial set of hotspots, wherein selection of a particular hotspot for inclusion in in the first subset is based at least in part on:
(i) the metastasis likelihood value calculated for the particular hotspot; and
(ii) one or more global hotspot features, each global hotspot feature determined using a plurality of hotspots in the initial set of hotspots; and
(g) causing, by the processor, rendering of a graphical representation of at least a portion of the first subset of hotspots for display within a graphical user interface (GUI).
20. The method of claim 19 , wherein the one or more global hotspot features comprises a total number of hotspots in the initial hotspot set.
21. The method of claim 20 , wherein step (f) comprises adjusting criteria for selection of hotspots for inclusion in the first subset based on the total number of hotspots in the initial hotspot set.
22. The method of claim 19 , wherein step (f) comprises using a machine learning module to select the first subset.
23. A method for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the method comprising:
(a) accessing, by a processor of a computing device, a bone scan image set for the human subject;
(b) automatically segmenting, by the processor, each image in the bone scan image set to identify one or more skeletal regions of interest, each skeletal region of interest corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images;
(c) automatically detecting, by the processor, an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images;
(d) for each hotspot in the initial set of hotspots, extracting, by the processor, a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculating, by the processor, a likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot;
(f) selecting, by the processor, a first subset of the hotspots of the initial set of hotspots based at least in part on the likelihood values calculated for each hotspot of the initial set of hotspots; and
(g) calculating, by the processor, one or more risk index values using at least a portion of the first subset of hotspots, said calculating comprising:
computing, for each particular hotspot of the portion of first subset, a skeletal involvement factor based on a ratio of (i) a size (e.g., area) of the particular hotspot to (ii) a size of a particular skeletal region to which the particular hotspot is assigned based on its location in the annotated set of images, thereby determining one or more skeletal involvement factors;
adjusting the skeletal involvement factors using one or more region-dependent correction factors, thereby obtaining one or more adjusted skeletal involvement factors; and
summing the adjusted skeletal involvement factors to determine the one or more risk index values.
24. The method of claim 23 , wherein, for each particular hotspot, the computed skeletal involvement factor estimates a proportion of total skeletal mass occupied by a physical volume associated with the particular hotspot.
25. The method of claim 24 , wherein the computing the skeletal involvement factor comprises:
calculating, by the processor, a ratio of an area of the particular hotspot to an area of the corresponding skeletal region of interest, thereby computing an area fraction for the particular hotspot; and
scaling the area fraction by a density coefficient associated with the skeletal region of interest to which the particular hotspot is assigned, thereby computing the skeletal involvement factor for the particular hotspot.
26. The method of claim 23 , wherein at least a portion of the hotspots of the first subset are assigned to a skeletal region of interest that is a member selected from the group consisting of a pelvis region, a lumbar region, and a sacrum region.
27. The method of claim 23 , wherein the one or more region-dependent correction factors comprise a sacrum region correction factor associated with a sacrum region and used to adjust skeletal involvement factors of hotspots identified as being located therein, and wherein the sacrum region correction factor has a value less than one.
28. The method of claim 23 , wherein the one or more region dependent correction factors comprise one or more correction factor pairs, each correction factor pair associated with a specific skeletal region of interest and comprising a first member and a second member (of the pair), wherein:
the first member of the pair is an anterior image correction factor and is used to adjust skeletal involvement factors computed for hotspots having been detected in an annotated anterior bone scan image of the annotated image set, and
the second member of the pair is a posterior image correction factor and is used to adjust skeletal involvement factors computed for hotspots having been detected in an annotated posterior bone scan image of the annotated image set.
29. A system for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the system comprising:
a processor; and
a memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to:
(a) access a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segment each image in the bone scan image set to identify one or more skeletal regions of interest, each corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images, wherein the one or more skeletal regions of interest comprise at least one of (i) and (ii):
(i) a femur region corresponding to a portion of a femur of the human subject, said femur portion encompassing at least three quarters of the femur along its length; and
(ii) a humerus region corresponding to a portion of a humerus of the human subject, said humerus portion encompassing at least three quarters of the humerus along its length;
(c) automatically detect an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images, said automatically detecting comprising identifying the one or more hotspots using intensities of pixels in the annotated set of images and using one or more region-dependent threshold values, and wherein the one or more region dependent threshold values include one or more values associated with the femur region and/or the humerus region that provide enhanced hotspot detection sensitivity in the femur region and/or the humerus region to compensate for reduced uptake of the agent therein;
(d) for each hotspot in the initial set of hotspots, extract a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculate a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot; and
(f) cause rendering of a graphical representation of at least a portion of the initial set of hotspots for display within a graphical user interface (GUI) and wherein, at step (b), the instructions cause the processor to:
compare each member of the bone scan image set with a corresponding atlas image of an atlas image set, each atlas image comprising one or more identifications of the one or more skeletal regions of interest, said skeletal regions of interest including (i) the femur region, comprising at least a portion of a knee region of the human subject, and/or (ii) the humerus region, comprising at least a portion of an elbow region of the human subject; and
for each image of the bone scan image set, register the corresponding atlas image with the image of the bone scan image set using the identified knee region and/or the identified elbow region in the image as (a) landmark(s), such that the identifications of the one or more skeletal regions of interest of the atlas image are applied to the image of the bone scan image set.
30. A method for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the method comprising:
(a) accessing, by a processor of a computing device, a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segmenting, by the processor, each image in the bone scan image set to identify one or more skeletal regions of interest, each corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images, wherein the one or more skeletal regions of interest comprise at least one of (i) and (ii):
(i) a femur region corresponding to a portion of a femur of the human subject, said femur portion encompassing at least three quarters of the femur along its length; and
(ii) a humerus region corresponding to a portion of a humerus of the human subject, said humerus portion encompassing at least three quarters of the humerus along its length;
(c) automatically detecting, by the processor, an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images, said automatically detecting comprising identifying the one or more hotspots using intensities of pixels in the annotated set of images and using one or more region-dependent threshold values, and wherein the one or more region dependent threshold values include one or more values associated with the femur region and/or the humerus region that provide enhanced hotspot detection sensitivity in the femur region and/or the humerus region to compensate for reduced uptake of the agent therein;
(d) for each hotspot in the initial set of hotspots, extracting, by the processor, a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculating, by the processor, a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot;
(f) causing, by the processor, rendering of a graphical representation of at least a portion of the initial set of hotspots for display within a graphical user interface (GUI);
(k) receiving, by the processor, via the GUI, a user selection of a second subset of the initial set of hotspots; and
(l) calculating, by the processor, one or more risk index values for the human subject based at least in part on a computed fraction of the skeleton of the human subject occupied by the second subset of hotspots.
31. A system for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the system comprising:
a processor; and
a memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to:
(a) access a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segment each image in the bone scan image set to identify one or more skeletal regions of interest, each skeletal region of interest corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images;
(c) automatically detect an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images, said automatically detecting comprising:
using (i) intensities of pixels in the annotated set of images and (ii) a plurality of preliminary threshold values to detect a set of potential hotspots;
computing a global threshold scaling factor using the set of potential hotspots;
adjusting the plurality of preliminary threshold values using the global threshold scaling factor, thereby obtaining a plurality of adjusted threshold values; and
using (i) intensities of pixels in the annotated set of images and (ii) the plurality of adjusted threshold values to identify the initial set of hotspots;
(d) for each hotspot in the initial set of hotspots, extract a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculate a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot; and
(f) cause rendering of a graphical representation of at least a portion of the initial set of hotspots for display within a graphical user interface (GUI).
32. A system for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the system comprising:
a processor; and
a memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to:
(a) access a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segment each image in the bone scan image set to identify one or more skeletal regions of interest, each skeletal region of interest corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images;
(c) automatically detect an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images;
(d) for each hotspot in the initial set of hotspots, extract a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculate a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot;
(f) select a first subset of the initial set of hotspots, wherein selection of a particular hotspot for inclusion in in the first subset is based at least in part on:
(i) the metastasis likelihood value calculated for the particular hotspot; and
(ii) one or more global hotspot features, each global hotspot feature determined using a plurality of hotspots in the initial set of hotspots; and
(g) cause rendering of a graphical representation of at least a portion of the first subset of hotspots for display within a graphical user interface (GUI).
33. A system for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the system comprising:
a processor; and
a memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to:
(a) access a bone scan image set for the human subject;
(b) automatically segment each image in the bone scan image set to identify one or more skeletal regions of interest, each skeletal region of interest corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images;
(c) automatically detect an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images;
(d) for each hotspot in the initial set of hotspots, extract a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculate a likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot;
(f) select a first subset of the hotspots of the initial set of hotspots based at least in part on the likelihood values calculated for each hotspot of the initial set of hotspots; and
(g) calculate one or more risk index values using at least a portion of the first subset of hotspots, said calculating comprising:
computing, for each particular hotspot of the portion of first subset, a skeletal involvement factor based on a ratio of (i) a size (e.g., area) of the particular hotspot to (ii) a size of a particular skeletal region to which the particular hotspot is assigned based on its location in the annotated set of images, thereby determining one or more skeletal involvement factors;
adjusting the skeletal involvement factors using one or more region-dependent correction factors, thereby obtaining one or more adjusted skeletal involvement factors; and
summing the adjusted skeletal involvement factors to determine the one or more risk index values.
34. A system for lesion marking and quantitative analysis of nuclear medicine images of a human subject, the system comprising:
a processor; and
a memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to:
(a) access a bone scan image set for the human subject, said bone scan image set obtained following administration of an agent to the human subject;
(b) automatically segment each image in the bone scan image set to identify one or more skeletal regions of interest, each corresponding to a particular anatomical region of a skeleton of the human subject, thereby obtaining an annotated set of images, wherein the one or more skeletal regions of interest comprise at least one of (i) and (ii):
(i) a femur region corresponding to a portion of a femur of the human subject, said femur portion encompassing at least three quarters of the femur along its length; and
(ii) a humerus region corresponding to a portion of a humerus of the human subject, said humerus portion encompassing at least three quarters of the humerus along its length;
(c) automatically detect an initial set of one or more hotspots, each hotspot corresponding to an area of elevated intensity in the annotated set of images, said automatically detecting comprising identifying the one or more hotspots using intensities of pixels in the annotated set of images and using one or more region-dependent threshold values, and wherein the one or more region dependent threshold values include one or more values associated with the femur region and/or the humerus region that provide enhanced hotspot detection sensitivity in the femur region and/or the humerus region to compensate for reduced uptake of the agent therein;
(d) for each hotspot in the initial set of hotspots, extract a set of hotspot features associated with the hotspot;
(e) for each hotspot in the initial set of hotspots, calculate a metastasis likelihood value corresponding to a likelihood of the hotspot representing a metastasis, based on the set of hotspot features associated with the hotspot;
(f) cause rendering of a graphical representation of at least a portion of the initial set of hotspots for display within a graphical user interface (GUI);
(k) receive via the GUI, a user selection of a second subset of the initial set of hotspots; and
(l) calculate one or more risk index values for the human subject based at least in part on a computed fraction of the skeleton of the human subject occupied by the second subset of hotspots.