IP Library Granted Patent US 12,014,488
Granted Patent B2
US 12,014,488 · App. 17/140,586 · Granted Jun 18, 2024

Distinguishing colon cancer stages based on computationally derived morphological features of cancer nuclei

Inventors: Anant Madabhushi (Shaker Heights, OH); Neeraj Kumar (Cleveland Heights, OH); Joseph E. Willis (Shaker Heights, OH)
Assignees: Case Western Reserve University; UH Cleveland Medical Center; The United States Government as Represente
G06T7/0012G06T7/11G06T7/41G06T7/62G16H10/40G16H30/40G16H50/20G16H70/60G06T2207/20081G06T2207/20084G06T2207/30024G06T2207/30028G06T2207/30081G06T2207/30096
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,014,488
App. No.
17/140,586
Granted
Jun 18, 2024
Kind
B2
Abstract

Embodiments discussed herein facilitate determination of cancer stages based at least in part on shape, size, and/or texture features of cancer nuclei. One example embodiment is a method, comprising: accessing at least a portion of a digital whole slide image (WSI) comprising a tumor; segmenting (e.g., via a first deep learning model) the tumor on the at least the portion of the digital WSI; segmenting (e.g., via a second deep learning model) cancer nuclei in the segmented tumor; extracting one or more features from the segmented cancer nuclei; providing the one or more features extracted from the segmented cancer nuclei to a trained machine learning model; and receiving, from the machine learning model, an indication of a cancer stage of the tumor.

Claims (53)

1. A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause a processor to perform operations, comprising:

accessing a segmented cancer nuclei in a segmented tumor from at least a portion of a digital whole slide image (WSI) comprising a tumor identified as having cancer present;

extracting a plurality of features from the segmented cancer nuclei, wherein the plurality of features comprise a standard deviation of nuclear area feature, an average nuclear area feature, an average major axis length feature, an average nuclei perimeter feature, and a kurtosis of nuclei area feature;

providing the plurality of features extracted from the segmented cancer nuclei to a trained machine learning model; and

receiving, from the trained machine learning model, an indication of a cancer stage of the tumor identified as having cancer present based upon the plurality of features extracted from the segmented cancer nuclei, wherein the indication of the cancer stage differentiates between different stages of cancer within the tumor identified as having cancer present.

2. The non-transitory computer-readable medium of claim 1 , wherein the operations further comprise:

generating a probability that the tumor is Stage 2 or Stage 4 based on the plurality of features; and

generating a classification of the tumor as Stage 2 or Stage 4 based, at least in part, on the probability.

3. The non-transitory computer-readable medium of claim 1 , wherein the plurality of features comprise at least one of: a statistic of at least one shape feature of the segmented cancer nuclei, the statistic of at least one size feature of the segmented cancer nuclei, or the statistic of at least one texture feature of the segmented cancer nuclei.

4. The non-transitory computer-readable medium of claim 3 , wherein the statistic is one of a mean, a median, a standard deviation, a skewness, a kurtosis, a range, a minimum, a maximum, a percentile, or histogram frequencies.

5. The non-transitory computer-readable medium of claim 3 , wherein the at least one size feature of the segmented cancer nuclei comprises one or more of an area, a major axis length, or a perimeter.

6. The non-transitory computer-readable medium of claim 1 , wherein the indication of the cancer stage of the tumor is a probability that the tumor is within one of the different stages of cancer.

7. The non-transitory computer-readable medium of claim 6 , wherein the operations further comprise:

segmenting the tumor on the at least the portion of the digital WSI comprises segmenting the tumor on the at least the portion of the digital WSI via a first deep learning (DL) model, wherein the first DL model is a fully convolutional deep neural network with a Visual Geometry Group (VGG)-18 architecture.

8. The non-transitory computer-readable medium of claim 6 , wherein the operations further comprise:

segmenting the tumor on the at least the portion of the digital WSI comprises segmenting the tumor on the at least the portion of the digital WSI via a first deep learning (DL) model, wherein the first DL model employs one of the following optimizers: an Adam optimizer, a stochastic gradient descent (SGD) optimizer, a SGD optimizer with momentum, or a SGD optimizer with Nesterov momentum.

9. The non-transitory computer-readable medium of claim 1 , wherein the operations further include:

segmenting the WSI to identify cancer within the WSI using a first deep learning (DL) model; and

segmenting the segmented cancer nuclei in the identified cancer using a second deep learning (DL) model that is different than the first DL model.

10. The non-transitory computer-readable medium of claim 9 , wherein the first DL model is a VGG-18 and the second DL model is a Mask-RCNN (Regional Convolutional Neural Network) with a Residual Network (Resnet)-50 architecture.

11. The non-transitory computer-readable medium of claim 9 , wherein the trained machine learning model distinguishes segmented cancer nuclei of a higher stage cancer as having a larger area, a larger perimeter, a larger length, and a greater variance in nuclear area than segmented cancer nuclei of a lower stage cancer, the higher stage cancer and the lower stage cancer being of the tumor identified as having cancer present.

12. The non-transitory computer-readable medium of claim 1 , wherein the segmented cancer nuclei comprises prostate cancer.

13. The non-transitory computer-readable medium of claim 12 , wherein the cancer stage is one of stage 2 or stage 4.

14. An apparatus, comprising:

memory configured to store at least a portion of a digital whole slide image (WSI) comprising a tumor identified as having cancer present;

one or more processors configured to perform operations comprising:

segmenting the tumor on the at least the portion of the digital WSI;

segmenting cancer nuclei in the segmented tumor;

extracting one or more features from the segmented cancer nuclei;

providing the one or more features extracted from the segmented cancer nuclei to a trained machine learning model;

receiving, from the trained machine learning model, an indication of a cancer stage of the tumor identified as having cancer present based upon the one or more features extracted from the segmented cancer nuclei, wherein the indication of the cancer stage differentiates between different stages of cancer within the tumor identified as having cancer present and wherein the indication of the cancer stage of the tumor is a probability that the tumor is within one of the cancer stages; and

wherein the trained machine learning model is configured to distinguish segmented cancer nuclei of a higher stage cancer as having a larger area, a larger perimeter, a larger length, and a greater variance in nuclear area than segmented cancer nuclei of a lower stage cancer, the higher stage cancer and the lower stage cancer being of the tumor identified as having cancer present.

15. The apparatus of claim 14 , wherein the one or more features comprise a standard deviation of nuclear area feature, an average nuclear area feature, an average major axis length feature, an average nuclei perimeter feature, and a kurtosis of nuclei area feature.

16. The apparatus of claim 14 , wherein the one or more features comprise at least one of: a statistic of at least one shape feature of the segmented cancer nuclei, the statistic of at least one size feature of the segmented cancer nuclei, or the statistic of at least one texture feature of the segmented cancer nuclei.

17. The apparatus of claim 16 , wherein the statistic is one of a mean, a median, a standard deviation, a skewness, a kurtosis, a range, a minimum, a maximum, a percentile, or histogram frequencies.

18. The apparatus of claim 16 , wherein the at least one size feature of the segmented cancer nuclei comprises one or more of an area, a major axis length, or a perimeter.

19. The apparatus of claim 14 , wherein segmenting the tumor on the at least the portion of the digital WSI comprises segmenting the tumor on the at least the portion of the digital WSI via a first deep learning (DL) model.

20. The apparatus of claim 14 , wherein segmenting cancer nuclei in the segmented tumor comprises segmenting cancer nuclei in the segmented tumor via a second deep learning (DL) model.

21. A method, comprising:

accessing a segmented cancer nuclei in a segmented colon cancer tumor from at least a portion of a digitized image having a tumor identified as having cancer present;

extracting one or more features from the segmented cancer nuclei;

providing the one or more extracted features extracted from the segmented cancer nuclei to a trained machine learning model; and

receiving, from the trained machine learning model, an indication of a cancer stage of the tumor identified as having cancer present based upon the one or more extracted features, wherein the indication of the cancer stage differentiates between different stages of cancer within the tumor identified as having cancer present; and

wherein the trained machine learning model is configured to distinguish segmented cancer nuclei of a higher stage cancer as having a larger area, a larger perimeter, a larger length, and a greater variance in nuclear area than segmented cancer nuclei of a lower stage cancer, the higher stage cancer and the lower stage cancer being of the tumor identified as having cancer present.

22. The method of claim 21 , wherein the one or more features comprise a standard deviation of nuclear area feature, an average nuclear area feature, an average major axis length feature, an average nuclei perimeter feature, and a kurtosis of nuclei area feature.

23. The method of claim 21 , wherein the trained machine learning model is configured to distinguish stage 2 colon cancer from stage 4 colon cancer.

24. The method of claim 21 , wherein the one or more features comprise at least one of: a statistic of at least one shape feature of the segmented cancer nuclei, the statistic of at least one size feature of the segmented cancer nuclei, or the statistic of at least one texture feature of the segmented cancer nuclei.

25. The method of claim 24 , wherein the indication of the cancer stage of the tumor is a probability that the tumor is within one of the different stages of cancer.

26. The method of claim 24 , wherein the at least one size feature of the segmented cancer nuclei comprises one or more of an area, a major axis length, or a perimeter.

27. The method of claim 21 , further comprising:

segmenting the digitized image to identify cancer within the digitized image using a first deep learning model; and

segmenting the segmented cancer nuclei in the identified cancer using a second deep learning model that is different than the first deep learning model.

28. The method of claim 27 , wherein the first deep learning model is a VGG-18 and the second deep learning model is a Mask-RCNN (Regional Convolutional Neural Network) with a Residual Network (Resnet)-50 architecture.

Assignments (4)
LICENSE Recorded Aug 8, 2024
From: CASE WESTERN RESERVE UNIVERSITY
To: NATIONAL INSTITUTES OF HEALTH
Reel/Frame 068218/0799 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2023
From: MADABHUSHI, ANANT
To: CASE WESTERN RESERVE UNIVERSITY; THE UNITED STATES GOVERNMENT AS REPRESENTED BY THE DEPARTMENT OF VETERAN AFFAIRS
Reel/Frame 064773/0302 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2023
From: KUMAR, NEERAJ
To: CASE WESTERN RESERVE UNIVERSITY
Reel/Frame 064773/0333 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2023
From: WILLIS, JOSEPH E.
To: CASE WESTERN RESERVE UNIVERSITY; UH CLEVELAND MEDICAL CENTER
Reel/Frame 064773/0415 →
Continuity (2)
Provisional Application 62984813 · Mar 4, 2020
Related Publication 20210279864A1 · Sep 9, 2021
Cited By (1)
US 12,562,279