IP Library Granted Patent US 12,444,044
Granted Patent B2
US 12,444,044 · App. 17/453,953 · Granted Oct 14, 2025

Artificial intelligence prediction of prostate cancer outcomes

Inventors: Craig Mermel (Los Gatos, CA); Yun Liu (Mountain View, CA); Naren Manoj (Mountain View, CA); Matthew Symonds (Mountain View, CA); Martin Stumpe (Mountain View, CA); Lily Peng (Mountain View, CA); Kunal Nagpal (Mountain View, CA); Ellery Wulczyn (Mountain View, CA); Davis Foote (Mountain View, CA); David F. Steiner (Mountain View, CA); Po-Hsuan Cameron Chen (Palo Alto, CA)
Assignee: Verily Life Sciences LLC
G06T7/0012A61B5/7275G06F18/21G06T2207/20081G06T2207/30024G06T2207/30081G06T2207/30096G06V2201/031
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,444,044
App. No.
17/453,953
Granted
Oct 14, 2025
Kind
B2
Abstract

One example method for AI prediction of prostate cancer outcomes involves receiving an image of prostate tissue; assigning Gleason pattern values to one or more regions within the image using an artificial intelligence Gleason grading model, the model trained to identify Gleason patterns on a patch-by-patch basis in a prostate tissue image; determining relative areal proportions of the Gleason patterns within the image; assigning at least one of a risk score or risk group value to the image based on the determined relative areal proportions; and outputting at least one of the risk score or the risk group value.

Claims (40)

1. A computer-implemented method of assessing a prognosis of a prostate cancer patient, comprising the steps of:

receiving an image of prostate tissue;

assigning Gleason pattern values to one or more regions within the image using an artificial intelligence Gleason grading model, the model trained to identify Gleason patterns on a patch-by-patch basis in a prostate tissue image;

determining relative areal proportions of the Gleason patterns within the image;

generating a continuous risk score for the entire image based on the determined relative areal proportions;

assigning at least one of the continuous risk score or risk group value, the risk group value based on the continuous risk score; and

outputting at least one of the continuous risk score or the risk group value.

2. The method of claim 1 , wherein the assigning at least one of the continuous risk score or the risk group value comprises using a Cox proportional hazard regression model fitted to a set of Gleason patterns or proportions thereof assigned to a multitude of images in a data set by the artificial intelligence Gleason grading model and associated survival data associated with the data set.

3. The method of claim 1 , wherein the image of the prostate tissue comprises an image of a specimen obtained from a prostatectomy or a needle core biopsy.

4. The method of claim 1 , further comprising presenting at least one of the risk score or the risk group value on a display of a workstation.

5. The method of claim 4 , further comprising receiving a Gleason Grade for the image from the workstation.

6. The method of claim 5 , further comprising combining the risk group value with a Gleason Grade.

7. The method of claim 6 , wherein combining the risk group value with the Gleason Grade comprises determining an average of the risk group value and the Gleason Grade.

8. The method of claim 1 , wherein the risk group values comprise integers from 1 to 5, inclusive, and wherein a survival probability of each of the risk group values approximately correspond to a survival probability of prostate cancer patients having Gleason Grades of integers 1-5 inclusive, respectively.

9. The method of claim 1 , further comprising using a model trained to distinguish prostate tissue from non-prostate tissue, wherein receiving the image comprises receiving only prostate tissue regions of the image from the model.

10. A system for assessing a prognosis of a prostate cancer patient, comprising:

a non-transitory computer-readable medium; and

one or more processors communicatively coupled to the non-transitory computer-readable medium, the one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

receive an image of prostate tissue;

assign Gleason pattern values to one or more regions within the image using an artificial intelligence Gleason grading model, the model trained to identify Gleason patterns on a patch-by-patch basis in a prostate tissue image;

determine relative areal proportions of the Gleason patterns within the image;

generate a continuous risk score for the entire image based on the determined relative areal proportions;

assign at least one of the continuous risk score or risk group value, the risk group value based on the continuous risk score; and

output at least one of the risk score or the risk group value.

11. The system of claim 10 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to use a Cox proportional hazard regression model fitted to a set of Gleason patterns or proportions thereof assigned to a multitude of images in a data set by the artificial intelligence Gleason grading model and associated survival data associated with the data set.

12. The system of claim 10 , wherein the image of the prostate tissue comprises an image of a specimen obtained from a prostatectomy or a needle core biopsy.

13. The system of claim 10 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to present at least one of the risk score or the risk group value on a display of a workstation.

14. The system of claim 13 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to receive a Gleason Grade for the image from the workstation.

15. The system of claim 14 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to combine the risk group value with a Gleason Grade.

16. The system of claim 15 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to determine an average of the risk group value and the Gleason Grade.

17. The system of claim 10 , wherein the risk group values comprise integers from 1 to 5, inclusive, and wherein a survival probability of each of the risk group values approximately corresponds to a survival probability of prostate cancer patients having Gleason Grades of integers 1-5 inclusive, respectively.

18. The system of claim 10 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to use a model trained to distinguish prostate tissue from non-prostate tissue, wherein receiving the image comprises receiving only prostate tissue regions of the image from the model.

19. A non-transitory computer-readable medium comprising processor-executable instructions configured to cause one or more processors to:

receive an image of prostate tissue;

assign Gleason pattern values to one or more regions within the image using an artificial intelligence Gleason grading model, the model trained to identify Gleason patterns on a patch-by-patch basis in a prostate tissue image;

determine relative areal proportions of the Gleason patterns within the image;

generate a continuous risk score for the entire image based on the determined relative areal proportions;

assign at least one of the continuous risk score or risk group value, the risk group value based on the continuous risk score; and

output at least one of the risk score or the risk group value.

20. The non-transitory computer-readable medium of claim 19 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to use a Cox proportional hazard regression model fitted to a set of Gleason patterns or proportions thereof assigned to a multitude of images in a data set by the artificial intelligence Gleason grading model and associated survival data associated with the data set.

Assignments (3)
CHANGE OF ADDRESS Recorded Nov 19, 2024
From: VERILY LIFE SCIENCES LLC
To: VERILY LIFE SCIENCES LLC
Reel/Frame 069390/0656 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2021
From: MERMEL, CRAIG; LIU, YUN; MANOJ, NAREN; SYMONDS, MATTHEW; STUMPE, MARTIN; PENG, LILY; NAGPAL, KUNAL; WULCZYN, ELLERY; FOOTE, DAVIS; STEINER, DAVID F.; CHEN, PO-HSUAN CAMERON
To: GOOGLE LLC
Reel/Frame 058073/0798 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2021
From: GOOGLE LLC
To: VERILY LIFE SCIENCES LLC
Reel/Frame 058073/0968 →
Continuity (2)
Provisional Application 63110786 · Nov 6, 2020
Related Publication 20220148169A1 · May 12, 2022
References Cited (31)
US 20070019854A1 · Gholap · 2007 [cited by examiner]
US 20130308849A1 · Fei · 2013 [cited by examiner]
US 20140233826A1 · Agaian · 2014 [cited by examiner]
US 20160350946A1 · Schieke · 2016 [cited by examiner]
US 20170032090A1 · Kamen · 2017 [cited by examiner]
US 20170073778A1 · Hoffmann et al. · 2017 [cited by applicant]
US 20170169276A1 · Agaian et al. · 2017 [cited by applicant]
US 20190340468A1 · Stumpe · 2019 [cited by examiner]
US 20190370965A1 · Lay · 2019 [cited by examiner]
US 20200015734A1 · Mayer · 2020 [cited by examiner]
US 20200263255A1 · Luca · 2020 [cited by examiner]
US 20210304405A1 · Cho · 2021 [cited by examiner]
WO 2019190518 · 2019 [cited by applicant]
Cox, D. R. Regression models and life-tables. in Springer Series in Statistics (eds Kotz, S., & Johnson, N. L.) 527-541 (Springer, New York, NY, 1992) (Year: 1992). [cited by examiner]
Arvaniti, Eirini, et al. “Automated Gleason grading of prostate cancer tissue microarrays via deep learning.” Scientific reports 8.1 (2018), pp. 1-11. (Year: 2018). [cited by examiner]
Nir, Guy, et al. “Comparison of artificial intelligence techniques to evaluate performance of a classifier for automatic grading of prostate cancer from digitized histopathologic images.” JAMA network open 2.3 (2019), p… [cited by examiner]
Ryu, Han Suk, et al. “Automated Gleason scoring and tumor quantification in prostate core needle biopsy images using deep neural networks and its comparison with pathologist-based assessment.” Cancers 11.12 (Nov. 25, 20… [cited by examiner]
Li, Yuchun, et al. “Automated gleason grading and gleason pattern region segmentation based on deep learning for pathological images of prostate cancer.” IEEE Access 8 (Jul. 7, 2020), pp. 117714-117725. (Year: 2020). [cited by examiner]
Nagpal, Kunal, et al. “Development and validation of a deep learning algorithm for Gleason grading of prostate cancer from biopsy specimens.” JAMA oncology 6.9 (Jul. 23, 2020), pp. 1372-1380 (Year: 2020). [cited by examiner]
Bulten et al., “Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study”, The Lancet Oncology 21.2 (2020): 233-241. [cited by applicant]
Cai et al., “Proxylessnas: Direct neural architecture search on target task and hardware”, arXiv preprint arXiv:1812.00332 (2018). [cited by applicant]
Chollet , “Xception: Deep learning with depthwise separable convolutions”, Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. [cited by applicant]
He , “Deep residual learning for image recognition”, Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. [cited by applicant]
Nagpal et al., “Development and validation of a deep learning algorithm for Gleason grading of prostate cancer from biopsy specimens”, JAMA oncology 6.9 (2020): 1372-1380. [cited by applicant]
Nagpal et al., “Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer”, NPJ digital medicine 2.1 (2019): 1-10. [cited by applicant]
Ström et al., “Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study”, The Lancet Oncology 21.2 (2020): 222-232. [cited by applicant]
Wulczyn et al., “A.I.-based Gleason Grading for Effective Stratification of Prostate Cancer Outcomes”, 2020, 26 pages. [cited by applicant]
Yamamoto et al., “Automated acquisition of explainable knowledge from unannotated histopathology images”, Nature communications 10.1 (2019): 1-9. [cited by applicant]
International Application No. PCT/US2021/072277 , International Search Report and Written Opinion, Mailed On Feb. 3, 2022, 10 pages. [cited by applicant]
EP21890346.6, “Extended European Search Report”, Sep. 20, 2024, 10 pages. [cited by applicant]
Pinckaers, et al., “Detection of Prostate Cancer in Whole-slide Images Through End-to-end Training With Image-level Labels”, Institute of Electrical and Electronics Engineers Transactions on Medical Imaging, vol. 40, No… [cited by applicant]