Systems and methods for predicting disease severity in ulcerative colitis
In some aspects, a method, a system, or a non-transitory computer-readable storage medium are described for training one or more models to predict ulcerative colitis (UC) severity based on human-interpretable image features extracted from a whole-slide image, including acts of accessing a plurality of annotated whole-slide images associated with a plurality of UC patients, wherein each of the plurality of annotated whole-slide images includes at least one annotation describing a cell-type label or a tissue-type segmentation for a portion of the whole-slide image, extracting a plurality of human-interpretable image features based on cell-type labels and tissue-type segmentations associated with the plurality of annotated whole-slide images, training a statistical model based on the plurality of human-interpretable image features to predict the UC severity for a whole-slide image, and storing the trained model on at least one storage device.
1 . A method for predicting a severity associated with an inflammatory bowel disease for a patient, the method comprising:
accessing a whole-slide image representing at least a portion of a colon biopsy obtained from the patient;
using a first statistical model to determine both one or more cell-type labels and one or more tissue-type segmentations associated with the whole-slide image;
determining a plurality of human-interpretable image features based on both the one or more cell-type labels and the one or more tissue-type segmentations associated with the whole-slide images, wherein the plurality of human-interpretable image features are selected based on a degree of correlation with consensus pathologist assessment according to Geboes scoring; and
using a second statistical model to predict a Geboes score associated with the inflammatory bowel disease based on the plurality of human-interpretable image features.
2 . The method of claim 1 , further comprising assessing a histological remission of the inflammatory bowel disease based on the Geboes score.
3 . The method of claim 1 , wherein the second statistical model comprises a neural network.
4 . The method of claim 1 , wherein the cell-type labels include one or more of: neutrophils, plasma cells, intraepithelial lymphocytes, non-intraepithelial lymphocytes, eosinophils, goblet cell nuclei, enterocytes, or a combination thereof.
5 . The method of claim 1 , wherein the tissue-type segmentations include one or more of: normal epithelium, neutrophil-infiltrated epithelium, goblet cell cytoplasm, crypt abscess, inter-gland lumen, blood vessels, lamina propria, muscularis mucosa erosion/ulceration, granulation tissue, or a combination thereof.
6 . The method of claim 1 , wherein the tissue-type segmentations includes basal plasmacytosis.
7 . The method of claim 1 , wherein the plurality of human-interpretable image features are reflective of active inflammation, epithelial injury, chronic inflammation, cell features related to neutrophils and resulting tissue features of active inflammation, and/or plasma cell features, and features of architectural changes.
8 . The method of claim 7 , wherein the plurality of human-interpretable image features comprises one or more of: combined area proportion of infiltrated epithelium (neutrophilic infiltration), crypt abscess, erosion, ulceration and associated granulation tissue over mucosa, area proportion of epithelium with neutrophilic infiltration over all epithelium, count proportion and density of neutrophils in epithelium, ulcer or their combination, goblet cell cytoplasm mucin area proportion in the epithelium, or a combination thereof.
9 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise one or more of: area proportion of erosion, ulceration and granulation tissue in mucosa, neutrophil density within infiltrated epithelium, crypt abscess, erosion, ulceration and granulation tissue, or a combination thereof.
10 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise one or more of: combined count proportions of chronic inflammatory cells (plasma cells, lymphocytes), or a combination thereof.
11 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise basal plasmacytosis area proportion in mucosa.
12 . The method of claim 7 , wherein the plurality of human-interpretable image features are reflective of one or more cell features related to neutrophils and/or resulting tissue features of active inflammation.
13 . The method of claim 7 , wherein the plurality of human-interpretable image features comprise one or more of: plasma cell count and density within mucosa, goblet cell cytoplasm mucin area proportion in an epithelium or a combination thereof.
14 . A system for predicting a severity associated with an inflammatory bowel disease (IBD) for a patient, the system comprising one or more processors configured to:
access a whole-slide image representing at least a portion of a colon biopsy obtained from the patient;
use a first statistical model to determine both one or more cell-type labels and one or more tissue-type segmentations associated with the whole-slide image;
determine a plurality of human-interpretable image features based on both the one or more cell-type labels and the one or more tissue-type segmentations associated with the whole-slide images, wherein the plurality of human-interpretable image features are selected based on a degree of correlation with consensus pathologist assessment according to Geboes scoring; and
use a second statistical model to predict a Geboes score associated with the inflammatory bowel disease based on the plurality of human-interpretable image features.
15 . The system of claim 14 , wherein the one or more processors are further configured to assess a histological remission of the inflammatory bowel disease based on the Geboes score.
16 . The system of claim 14 , wherein the second statistical model comprises a neural network.
17 . The system of claim 14 , wherein the cell-type labels include one or more of:
neutrophils, plasma cells, intraepithelial lymphocytes, non-intraepithelial lymphocytes, eosinophils, goblet cell nuclei, enterocytes, or a combination thereof.
18 . The system of claim 14 , wherein the tissue-type segmentations include one or more of: normal epithelium, neutrophil-infiltrated epithelium, goblet cell cytoplasm, crypt abscess, inter-gland lumen, blood vessels, lamina propria, muscularis mucosa erosion/ulceration, granulation tissue, or a combination thereof.
19 . The system of claim 14 , wherein the tissue-type segmentations includes basal plasmacytosis.
20 . The system of claim 14 , wherein the plurality of human-interpretable image features are reflective of active inflammation, epithelial injury, chronic inflammation, cell features related to neutrophils and resulting tissue features of active inflammation, and/or plasma cell features, and features of architectural changes.
21 . The system of claim 20 , wherein the plurality of human-interpretable image features comprises one or more of: combined area proportion of infiltrated epithelium (neutrophilic infiltration), crypt abscess, erosion, ulceration and associated granulation tissue over mucosa, area proportion of epithelium with neutrophilic infiltration over all epithelium, count proportion and density of neutrophils in epithelium, ulcer or their combination, goblet cell cytoplasm mucin area proportion in the epithelium, or a combination thereof.
22 . The system of claim 20 , wherein the plurality of human-interpretable image features comprise one or more of: area proportion of erosion, ulceration and granulation tissue in mucosa, neutrophil density within infiltrated epithelium, crypt abscess, erosion, ulceration and granulation tissue, or a combination thereof.
23 . The system of claim 20 , wherein the plurality of human-interpretable image features comprise one or more of: combined count proportions of chronic inflammatory cells (plasma cells, lymphocytes), or a combination thereof.
24 . The system of claim 20 , wherein the plurality of human-interpretable image features comprise basal plasmacytosis area proportion in mucosa.
25 . The system of claim 20 , wherein the plurality of human-interpretable image features are reflective of one or more cell features related to neutrophils and/or resulting tissue features of active inflammation.