IP Library › Granted Patent US 12,670,595
Granted Patent B1
US 12,670,595 · App. 18/809,083 · Granted Jun 30, 2026

Systems and methods for predicting disease severity in ulcerative colitis

Inventors: Fedaa Najdawi (Brookline, MA); Kathleen Sucipto (Brookline, MA); Archit Khosla (New York, NY); Michael Drage (Wellesley, MA); Amaro N. Taylor-Weiner (Brooklyn, NY); Michael C. Montalto (Brielle, NJ); Murray Resnick (Sharon, MA); Maryam Pouryahya (Bethesda, MD); Stephanie Hennek (Medford, MA); Ilan N. Wapinski (Brookline, MA); Andrew H. Beck (Brookline, MA); Christina Jayson (Somerville, MA); Chintan Shah (Chestnut Hill, MA); Waleed Tahir (Dorchester, MA); John Shamshoian (Somerville, MA); Michael Griffin (Georgetown, TX); Lani Clinton (Efland, NC); Zahil Shanis (Claymont, DE); Carlos Gaitán (Kirkland, WA); Jin Li (New York, NY); George Hu (Boston, MA); Andrew Walker (Hopkins, MN); Harshith Padigela (Brookline, MA); Harsha Vardhan Pokkalla (Sudbury, MA); Yibo Zhang (Lynnfield, MA); Emma Krause (Newburyport, MA); Jimish Mehta (Ardmore, PA)
Assignee: PathAI, Inc.
G06T7/0012G06T7/11G06T2207/20081G06T2207/30024G06T2207/30028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,670,595
App. No.
18/809,083
Filed
Aug 19, 2024
Granted
Jun 30, 2026
Kind
B1
Art Unit
2661
USPC
382/128
Abstract

In some aspects, a method, a system, or a non-transitory computer-readable storage medium are described for training one or more models to predict ulcerative colitis (UC) severity based on human-interpretable image features extracted from a whole-slide image, including acts of accessing a plurality of annotated whole-slide images associated with a plurality of UC patients, wherein each of the plurality of annotated whole-slide images includes at least one annotation describing a cell-type label or a tissue-type segmentation for a portion of the whole-slide image, extracting a plurality of human-interpretable image features based on cell-type labels and tissue-type segmentations associated with the plurality of annotated whole-slide images, training a statistical model based on the plurality of human-interpretable image features to predict the UC severity for a whole-slide image, and storing the trained model on at least one storage device.

Claims (34)

1 . A method for predicting a severity associated with an inflammatory bowel disease for a patient, the method comprising:

accessing a whole-slide image representing at least a portion of a colon biopsy obtained from the patient;

using a first statistical model to determine both one or more cell-type labels and one or more tissue-type segmentations associated with the whole-slide image;

determining a plurality of human-interpretable image features based on both the one or more cell-type labels and the one or more tissue-type segmentations associated with the whole-slide images, wherein the plurality of human-interpretable image features are selected based on a degree of correlation with consensus pathologist assessment according to Geboes scoring; and

using a second statistical model to predict a Geboes score associated with the inflammatory bowel disease based on the plurality of human-interpretable image features.

2 . The method of claim 1 , further comprising assessing a histological remission of the inflammatory bowel disease based on the Geboes score.

3 . The method of claim 1 , wherein the second statistical model comprises a neural network.

4 . The method of claim 1 , wherein the cell-type labels include one or more of: neutrophils, plasma cells, intraepithelial lymphocytes, non-intraepithelial lymphocytes, eosinophils, goblet cell nuclei, enterocytes, or a combination thereof.

5 . The method of claim 1 , wherein the tissue-type segmentations include one or more of: normal epithelium, neutrophil-infiltrated epithelium, goblet cell cytoplasm, crypt abscess, inter-gland lumen, blood vessels, lamina propria, muscularis mucosa erosion/ulceration, granulation tissue, or a combination thereof.

6 . The method of claim 1 , wherein the tissue-type segmentations includes basal plasmacytosis.

7 . The method of claim 1 , wherein the plurality of human-interpretable image features are reflective of active inflammation, epithelial injury, chronic inflammation, cell features related to neutrophils and resulting tissue features of active inflammation, and/or plasma cell features, and features of architectural changes.

8 . The method of claim 7 , wherein the plurality of human-interpretable image features comprises one or more of: combined area proportion of infiltrated epithelium (neutrophilic infiltration), crypt abscess, erosion, ulceration and associated granulation tissue over mucosa, area proportion of epithelium with neutrophilic infiltration over all epithelium, count proportion and density of neutrophils in epithelium, ulcer or their combination, goblet cell cytoplasm mucin area proportion in the epithelium, or a combination thereof.

9 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise one or more of: area proportion of erosion, ulceration and granulation tissue in mucosa, neutrophil density within infiltrated epithelium, crypt abscess, erosion, ulceration and granulation tissue, or a combination thereof.

10 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise one or more of: combined count proportions of chronic inflammatory cells (plasma cells, lymphocytes), or a combination thereof.

11 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise basal plasmacytosis area proportion in mucosa.

12 . The method of claim 7 , wherein the plurality of human-interpretable image features are reflective of one or more cell features related to neutrophils and/or resulting tissue features of active inflammation.

13 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise one or more of: plasma cell count and density within mucosa, goblet cell cytoplasm mucin area proportion in an epithelium or a combination thereof.

14 . A system for predicting a severity associated with an inflammatory bowel disease (IBD) for a patient, the system comprising one or more processors configured to:

access a whole-slide image representing at least a portion of a colon biopsy obtained from the patient;

use a first statistical model to determine both one or more cell-type labels and one or more tissue-type segmentations associated with the whole-slide image;

determine a plurality of human-interpretable image features based on both the one or more cell-type labels and the one or more tissue-type segmentations associated with the whole-slide images, wherein the plurality of human-interpretable image features are selected based on a degree of correlation with consensus pathologist assessment according to Geboes scoring; and

use a second statistical model to predict a Geboes score associated with the inflammatory bowel disease based on the plurality of human-interpretable image features.

15 . The system of claim 14 , wherein the one or more processors are further configured to assess a histological remission of the inflammatory bowel disease based on the Geboes score.

16 . The system of claim 14 , wherein the second statistical model comprises a neural network.

17 . The system of claim 14 , wherein the cell-type labels include one or more of:

neutrophils, plasma cells, intraepithelial lymphocytes, non-intraepithelial lymphocytes, eosinophils, goblet cell nuclei, enterocytes, or a combination thereof.

18 . The system of claim 14 , wherein the tissue-type segmentations include one or more of: normal epithelium, neutrophil-infiltrated epithelium, goblet cell cytoplasm, crypt abscess, inter-gland lumen, blood vessels, lamina propria, muscularis mucosa erosion/ulceration, granulation tissue, or a combination thereof.

19 . The system of claim 14 , wherein the tissue-type segmentations includes basal plasmacytosis.

20 . The system of claim 14 , wherein the plurality of human-interpretable image features are reflective of active inflammation, epithelial injury, chronic inflammation, cell features related to neutrophils and resulting tissue features of active inflammation, and/or plasma cell features, and features of architectural changes.

21 . The system of claim 20 , wherein the plurality of human-interpretable image features comprises one or more of: combined area proportion of infiltrated epithelium (neutrophilic infiltration), crypt abscess, erosion, ulceration and associated granulation tissue over mucosa, area proportion of epithelium with neutrophilic infiltration over all epithelium, count proportion and density of neutrophils in epithelium, ulcer or their combination, goblet cell cytoplasm mucin area proportion in the epithelium, or a combination thereof.

22 . The system of claim 20 , wherein the plurality of human-interpretable image features comprise one or more of: area proportion of erosion, ulceration and granulation tissue in mucosa, neutrophil density within infiltrated epithelium, crypt abscess, erosion, ulceration and granulation tissue, or a combination thereof.

23 . The system of claim 20 , wherein the plurality of human-interpretable image features comprise one or more of: combined count proportions of chronic inflammatory cells (plasma cells, lymphocytes), or a combination thereof.

24 . The system of claim 20 , wherein the plurality of human-interpretable image features comprise basal plasmacytosis area proportion in mucosa.

25 . The system of claim 20 , wherein the plurality of human-interpretable image features are reflective of one or more cell features related to neutrophils and/or resulting tissue features of active inflammation.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Jul 28, 2026
From: ORBIMED ROYALTY & CREDIT OPPORTUNITIES IV, LP, AS ADMINISTRATIVE AGENT
To: PATHAI, INC.
Reel/Frame 075427/0613 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 17, 2025
From: NAJDAWI, FEDAA; SUCIPTO, KATHLEEN; KHOSLA, ARCHIT; DRAGE, MICHAEL; TAYLOR-WEINER, AMARO N.; MONTALTO, MICHAEL C.; RESNICK, MURRAY; POURYAHYA, MARYAM; WAPINSKI, ILAN N.
To: PATHAI, INC.
Reel/Frame 073250/0118 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 17, 2025
From: JAYSON, CHRISTINA
To: PATHAI, INC.
Reel/Frame 073250/0233 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 17, 2025
From: CLINTON, LANI
To: PATHAI, INC.
Reel/Frame 073250/0474 →
EMPLOYMENT AGREEMENT Recorded Dec 17, 2025
From: GAITÁN, CARLOS
To: PATHAI, INC.
Reel/Frame 074009/0587 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 17, 2025
From: HENNEK, STEPHANIE; BECK, ANDREW H.; SHAH, CHINTAN; TAHIR, WALEED; SHAMSHOIAN, JOHN; GRIFFIN, MICHAEL; SHANIS, ZAHIL; LI, JIN; HU, GEORGE; WALKER, ANDREW; PADIGELA, HARSHITH; POKKALLA, HARSHA VARDHAN; ZHANG, YIBO; KRAUSE, EMMA; MEHTA, JIMISH
To: PATHAI, INC.
Reel/Frame 074810/0281 →
Continuity (4)
Continuation 17955301 · Sep 28, 2022
Continuation In Part 17375694 · Jul 14, 2021
Provisional Application 63252506 · Oct 5, 2021
Provisional Application 63052316 · Jul 15, 2020
References Cited (71)
US 8712142B2 · Rajpoot · 2014 [cited by examiner]
US 9873742B2 · Keir · 2018 [cited by applicant]
US 12048414B2 · Sasada · 2024 [cited by examiner]
US 20060233797A1 · Gujrathi · 2006 [cited by examiner]
US 20150294459A1 · Rutenberg · 2015 [cited by examiner]
US 20170076448A1 · Chen · 2017 [cited by examiner]
US 20190108912A1 · Spurlock · 2019 [cited by applicant]
US 20200208114A1 · Baryawno et al. · 2020 [cited by applicant]
US 20210056688A1 · Xu et al. · 2021 [cited by applicant]
US 20220036971A1 · Yoo et al. · 2022 [cited by applicant]
US 20220058776A1 · Ozcan et al. · 2022 [cited by applicant]
US 20220108123A1 · Bhargava et al. · 2022 [cited by applicant]
CN 108471949A · 2018 [cited by applicant]
WO WO2019025533A1 · 2019 [cited by applicant]
WO 2021257710A1 · 2021 [cited by applicant]
WO 2022093510A1 · 2022 [cited by applicant]
Bass, J.A., Friesen, C.A., Deacy, A.D. et al. Investigation of potential early Histologic markers of pediatric inflammatory bowel disease. BMC Gastroenterol 15, 129 (2015). https://doi.org/10.1186/s12876-015-0359-2 (Yea… [cited by examiner]
Geboes K, Riddell R, Öst A, et alA reproducible grading scale for histological assessment of inflammation in ulcerative colitisGut 2000;47:404-409 (Year: 2000). [cited by examiner]
Campanella et al., Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine. Aug. 2019; 25:1301-1309. [cited by applicant]
Coudray et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nature Medicine. Oct. 2018; 24:1559-1567. [cited by applicant]
Chaddad et al, Predicting Gleason Score of Prostate Cancer Patients Using Radiomic Analysis. Radiomic Analysis. Frontiers in Oncology. Dec. 18, 2018;8(630). 10 Pages. [cited by applicant]
Ehteshami Bejnordi et al., Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA. 2017;318 (22):2199-2210. [cited by applicant]
He et al., Histology image analysis for carcinoma detection and grading. Computer Methods and Programs in Biomedicine. 2012; 107:538-556. [cited by applicant]
Kalra et al., Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence. NPJ Digital Medicine. 2020; 3:31. [cited by applicant]
Khosravi et al., Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images. EbioMedicine. 2018; 27:317-328. [cited by applicant]
Korbar et al., Deep Learning for Classification of Colorectal Polyps on Whole-slide Images. Journal of Pathology Informatics. Jul. 25, 2017; 8:30. [cited by applicant]
Madabhushi et al., Image analysis and machine learning in digital pathology: Challenges and opportunities. Medical Image Analysis. 2016; 33:170-175. [cited by applicant]
McCann et al., Automated Colitis Detection From Endoscopic Biopsies As A Tissue Screening Tool in Diagnostic Pathology. 19 [cited by applicant]
Nagpal et al., Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digital Medicine. 2019: 2:48. [cited by applicant]
Rostam et al., Image based Machine Learning for identification of macrophage subsets. Scientific Reports. Jun. 14, 2017; 7:3521. [cited by applicant]
Sadimin et al., Pathology Imaging Informatics for Clinical Practice and Investigative and Translational Research. N. Am. J. Med. Sci. Apr. 2012; 5(2):103-109. [cited by applicant]
Saltz et al., Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. Cell Reports. Apr. 3, 2018; 23:181-200. [cited by applicant]
Sha et al., Multi-Field-of-View Deep Learning Model Predicts Nonsmall Cell Lung Cancer Programmed Death-Ligand 1 Status from Whole-Slide Hematoxylin and Eosin Images. Journal of Pathology Informatics. Jul. 23, 2019; 10:… [cited by applicant]
Thorstenson et al., Implementation of large-scale routine diagnostics using whole slide imaging in Sweden: Digital pathology experiences 2006-2013. Journal of Pathology Informatics. Mar. 28, 2014; 5:14. [cited by applicant]
Wang et al., Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome. Scientific Reports. Jul. 10, 2018; 8:10393. [cited by applicant]
Wang et al., Deep Learning for Identifying Metastatic Breast Cancer. arXiv [q-bio.QM] (2016). [cited by applicant]
Yu et al., Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nature Communications. Aug. 16, 2016; 7:12474. [cited by applicant]
Althammer et al., Automated image analysis of NSCLC biopsies to predict response to anti-PD-L1 therapy. Research Article. Journal for ImmunoTherapy of Cancer. 2019;7:121. 12 Pages. [cited by applicant]
Janowczyk et al., Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. Journal of Pathology Informatics. Jul. 26, 2016;1:29. 18 Pages. [cited by applicant]
Wang et al., Computational Staining of Pathology Images to Study the Tumor Microenvironment in Lung Cancer. Cancer Research. 2020;80(10):2056-66. [cited by applicant]
Zarella et al., A Practical Guide to Whole Slide Imaging: A White Paper From the Digital Pathology Association. Archives of Pathology & Laboratory Medicine. College of American Pathologists. Oct. 11, 2018;143(2):222-34. [cited by applicant]
[No. Author Listed], What is OpenPolScope? 2019. https://openpolscope.org/ [Last accessed Dec. 6, 2023]. [cited by applicant]
Almangush et al., Stromal categorization in early oral tongue cancer. Virchows Archiv.2021;478:925-32. [cited by applicant]
Carrasco-Zevallos et al., AI-based histologic measurement of NASH (AIM-NASH): A drug development tool for assessing clinical tiral endpoints. Oral Presentation at the International Liver Congree. Jun. 23-26, 2021. [cited by applicant]
Chandler et al., The double edge sword of fibrosis in cancer. Transl Res. Jul. 2019;209:55-67. doi: 10.1016/j.trsl.2019.02.006. Epub Feb. 2, 20191. [cited by applicant]
Conway et al., Generation of an atlas characterizing the tumor immune microenvironment via AI-based histologic mapping of multiple cancer types at scale. Poster presented at Pathology Visions. Oct. 15-18, 2022. 1 Page. [cited by applicant]
Diao et al., Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes. Nature Communications. 2021;16:1613. 15 Pages. [cited by applicant]
Ding et al., Image analysis reveals molecularly distinct patterns of TILs in NSCLC associated with treatment outcome. Precision Oncology.2022;6:33.15 Pages. [cited by applicant]
Duanmu et al., Spatial Attention-Based Deep Learning System for Breast Cancer Pathological Complete Response Prediction with Serial Histopathology Images in Multiple Stains. HHS Public Access. NCBI. Oct. 6, 2022. 14 Pag… [cited by applicant]
Esbona et al. The Presence of Cyclooxygenase 2, Tumor-Associated Macrophages, and Collagen Alignment as Prognostic Markers for Invasive Breast Carcinoma Patients. Am J Pathol. Mar. 2018;188(3):559-573. doi: 10.1016/j.aj… [cited by applicant]
Fanous et al., Quantitative phase imaging of stromal prognostic markers in pancreatic ductal adenocarcinoma. Biomed Opt Express. Feb. 1, 20202;11(3):1354-1364. doi: 10.1364/BOE.383242. [cited by applicant]
Fereidouni et al., Dual-mode emission and transmission microscopy for virtual histochemistry using hematoxylin-and eosin-stained tissue sections. Biomedical Optics Express. Dec. 1, 2019;10(12):6516-30. [cited by applicant]
Grunwald et al., Spatially confined sub-tumor microenvironments in pancreatic cancer. Cell. Oct. 2, 20218;184(22):5577-5592.e18. doi: 10.1016/j.cell.2021.09.022. Epub Oct. 1, 20212. PMID: 34644529. [cited by applicant]
Hanley et al., A subset of myofibroblastic cancer-associated fibroblasts regulate collagen fiber elongation, which is prognostic in multiple cancers. Oncotarget. Feb. 2, 2016;7(5):6159-74. doi: 10.18632/oncotarget.6740. [cited by applicant]
Janjigian et al., Pack Years of Cigarette Smoking as a Prognostic Factor in Patients with Stage IIIB/IV Non-Small Cell Lung Cancer.Cancer.Feb. 1, 2010;116(3):670-75. [cited by applicant]
Kandel et al., Label-free tissue scanner for colorectal cancer screening. Journal of Biomedical Optics. Jun. 2017;22(6). 15 Pages. [cited by applicant]
Marini et al., H&E-adversarial network: a convolutional neural network to learn stain-invariant features through Hemxatoxylin & Eosin regression. arxiv.org. Jan. 19, 2022. 10 Pages. [cited by applicant]
Michener et al., AI-Powered analysis of nuclear morphology associated with prognosis in high-grade serous carcinoma. Poster presented at ESMO Congress 2022. Sep. 9-13, 2022. (Abstract 5252). [cited by applicant]
Nearchou et al., Automated Detection and Classification of Desmoplastic Reaction at the Colorectal Tumour Front Using Deep Learning. Cancers.Mar. 2, 20216:13(1615). 15 pages. [cited by applicant]
Nost et al., Time trends of persistent organic pollutants in 30 year old sampled in 1986, 1994,2001 and 2007 in Northern Norway: Measurements, mechanistic modeling and a comparison of study designs. Environmental Resear… [cited by applicant]
Ouellette et al., Navigating the Collagen Jungle: The Biomedical Potential of Fiber Organization in Cancer. Bioengineering.Jan. 21, 2021;8(17). 19 Pages. [cited by applicant]
Patil et al., Intratumoral plasma cells predict outcomes to PD-L1 blockade in non-small cell lung cancer. Cancer Cell. Mar. 1, 20224;40:1-12. [cited by applicant]
Rajan et al., Deep-Learning Based Prediction of c-Met Status From Digitized H&E-Stained Non-Small Cell Lung Cancer Tissue Samples. Poster Presented at the IASLC World Conference on Lung Cancer. Aug. 6-9, 2022. (Abstract… [cited by applicant]
Salmon et al., Matrix architecture defines the preferential localization and migration of T cells into the stroma of human lung tumors. J Clin Invest. Mar. 2012;122(3):899-910. doi: 10.1172/JCI45817. Epub Feb. 1, 2012. [cited by applicant]
Simon et al., Cancer-Associated Fibroblast Subpopulations With Diverse and Dynamic Roles in the Tumor Microenvironment. Mol Cancer Res. Feb. 2022;20(2):183-192. doi: 10.1158/1541-7786.MCR-21-0282. Epub Oct. 20, 2021. [cited by applicant]
Sun et al., Tumour DDR1 promotes collagen fiber alignment to instigate immune exclusion. Nature.2021;599:673-78. [cited by applicant]
Takahasi et al., Fibrous Stroma Is Associated with Poorer Prognosis in Lung Squamous Cell Carcinoma Patients. Journal of Thoracic Oncology. Sep. 9, 2011;6(9): 1460-67. [cited by applicant]
Ueno et al., Histopathological atlas of desmoplastic reaction characterization in colorectal cancer. Jpn J Clin Oncol. May 2, 20218;51(6):1004-1012. doi: 10.1093/jjco/hyab040. [cited by applicant]
Wang et al., Smoking History as a Potential Predictor of Immune Checkpoint Inhibitor Efficacy in Metastatic Non-Small Cell Lung Cancer. J Natl Cancer Inst. Nov. 2, 20219;113(12):1761-69. [cited by applicant]
Yanai et al., Presence of myxoid stromal change and fibrotic focus in pathological examination are prognostic factors of triple-negative breast cancer: Results from a retrospective single center study. 2021. PLoS One 16… [cited by applicant]
Zhang et al., Quantitative multimodal anisotropy imagine enables automated fibrosis assessment of H&E-stained tissue, Poster Presented at the International Liver Congress.Jun. 22-26, 2022. [cited by applicant]