Weakly-supervised system, method and workflow for processing whole slide image for disease detection
In generating training and testing datasets for machine learning, manual annotation of a large volume of data is impractical and expensive. A machine-learning model for detecting carcinoma (CA) from a whole slide image (WSI) processes average cellular features of cells identified on the WSI. Each cellular feature is a descriptive statistic of the cells, advantageously allowing the training and testing datasets to be constructed without a costly annotation process of pixelwise labelling each cell on a WSI training sample. Apart from predicting a CA case or a non-CA case for the WSI, the machine-learning model is also usable to: identify a suspicious CA case for priority assessment if a non-CA case is predicted for the WSI; generate a tumor probability heatmap of the WSI for visualizing potential CA regions on the WSI to assist pathological assessment; and assess quality control of a triage system before implementation in clinical setting.
1 . A computer-implemented method for processing a whole slide image (WSI) to detect carcinoma (CA), the method comprising:
setting up a machine-learning model for classifying the WSI as a CA case or as a non-CA case, the machine-learning model being realized as a plurality of ensembled networks with a classification decision made by the machine-learning model according to a plurality of probabilities of having malignancy respectively generated by the plurality of ensembled networks, an individual ensembled network being realized as a multilayer perception (MLP) network configured by a plurality of hyperparameters, the plurality of hyperparameters being learnable, wherein the machine-learning model is arranged to process a plurality of averaged cellular features for CA detection, and wherein an individual averaged cellular feature is a descriptive statistic of cells identified on the WSI, allowing a training dataset for training the machine-learning model and a testing dataset for verifying the trained machine-learning model to be constructed without a need to involve a costly annotation process of pixelwise labelling each cell on a WSI training sample; and
using the training and testing datasets to learn respective pluralities of hyperparameters for the plurality of ensembled networks and to train the plurality of ensembled networks.
2 . The method of claim 1 , wherein a plurality of descriptive statistics forming the plurality of averaged cellular features includes a plurality of statistical parameters regarding geometric dimensions of the identified cells and regarding optical densities of staining reagents applied to the identified cells.
3 . The method of claim 1 , wherein the plurality of hyperparameters is selected from a group consisting of a choice of activation function, a choice of L2 regulation term, a dropout rate, a number of hidden layers, a shape of the hidden layers, a choice of last-neuron activation function, a choice of optimization algorithm, a choice of loss function, an epoch size and a batch size.
4 . The method of claim 1 , wherein:
the plurality of ensembled networks consists of a predetermined number of ensembled networks; and
the respective pluralities of hyperparameters as learnt are optimized pluralities of hyperparameters such that the plurality of ensembled networks consists of the predetermined number of best-performing ensembled networks over a plurality of ensembled-network candidates contending for inclusion in the plurality of ensembled networks.
5 . The method of claim 4 , wherein the predetermined number is selected to be an odd number.
6 . The method of claim 1 , wherein the classification decision made by the machine-learning model is a majority vote of respective classification decisions made by the plurality of ensembled networks, and wherein the individual ensembled network makes a corresponding classification decision according to a corresponding probability of having malignancy generated by the individual ensembled network.
7 . The method of claim 1 , wherein the machine-learning model makes the classification decision according to an average probability of having malignancy over the plurality of ensembled networks.
8 . The method of claim 1 further comprising predicting the CA case or the non-CA case from the WSI, wherein the predicting of the CA case or the non-CA case from the WSI comprises:
identifying a plurality of cells on the WSI;
extracting a plurality of cellular features of different types for an individual cell, whereby respective pluralities of cellular features of different types are obtained for the plurality of cells;
down-sampling the respective pluralities of cellular features of different types into the plurality of averaged cellular features, wherein an individual average cellular feature is obtained by averaging cellular features of a corresponding type in the respective pluralities of cellular features of different types; and
using the trained machine-learning model to process the plurality of averaged cellular features to predict whether the plurality of cells identified on the WSI constitutes the CA case or the non-CA case.
9 . The method of claim 8 , wherein the identifying of the plurality of cells on the WSI comprises using a watershed algorithm to segment the WSI into the plurality of cells.
10 . The method of claim 8 , wherein the identifying of the plurality of cells on the WSI comprises using a convolutional neural network (CNN) to segment the WSI into the plurality of cells after the CNN is trained.
11 . The method of claim 8 , wherein in using the trained machine-learning model to process the plurality of averaged cellular features to predict the CA case or the non-CA case, the machine-learning model computes an average probability of having malignancy over the plurality of ensembled networks as a malignancy prediction score for indicating a likelihood of presence of potential CA cells for the WSI, the average probability of having malignancy being computed according to the plurality of probabilities of having malignancy generated by the plurality of ensembled networks.
12 . The method of claim 11 further comprising prioritizing a suspicious CA case from the WSI after the non-CA case is predicted for the WSI, wherein the prioritizing of the suspicious CA case from the WSI comprises:
collecting a plurality of checked WSIs, an individual checked WSI being predicted to be the non-CA case;
tiling the WSI and the plurality of checked WSIs to form a composite WSI;
extracting a second plurality of average cellular features for the composite WSI;
using the trained machine-learning model to process the second plurality of average cellular features instead of the plurality of average cellular features to thereby predict the CA case or the non-CA case for the composite WSI;
responsive to predicting that the composite WSI is the CA case, changing a classification of the WSI from the non-CA case to the suspicious CA case; and
responsive to classifying the WSI as the suspicious CA case, triaging the WSI for priority assessment of CA.
13 . A computer-implemented method for predicting one or more quality-control-related parameters in running a triage system used for diagnosing carcinoma (CA) cases, the method comprising:
preparing or acquiring a plurality of whole slide images (WSIs) for CA detection, wherein an individual WSI contains a plurality of cells;
processing the individual WSI to detect CA according to the method of claim 12 to thereby classify the individual WSI as a CA case, a non-CA case or a suspicious CA case;
repeating the processing of the individual WSI for the plurality of WSIs so as to divide the plurality of WSIs into a first plurality of WSIs classified as CA cases, a second plurality of WSIs classified as non-CA cases, and a third plurality of WSIs classified as suspicious CA cases; and
simulating the triage system with the first, second and third pluralities of classified WSIs as inputs to the triage system to predict the one or more quality-control-related parameters.
14 . The method of claim 13 , wherein a first quality-control-related parameter selected from the one or more quality-control-related parameters is a percentage of non-CA cases for respective non-CA cases to be skipped by using the machine learning or deep learning algorithm-based triage system against under no case prioritization.
15 . The method of claim 13 , wherein a second quality-control-related parameter selected from the one or more quality-control-related parameters is a possible time saved in diagnosing the plurality of WSIs by using the triage system against under no case prioritization.
16 . The method of claim 13 , wherein a third quality-control-related parameter selected from the one or more quality-control-related parameters is a percentage of time for respective non-CA cases to be skipped by a pathologist or a related medical professional.
17 . The method of claim 1 further comprising generating a tumor probability heatmap of the WSI for facilitating visualization of potential CA regions on the WSI to assist pathological assessment of the WSI, wherein the generating of the tumor probability heatmap comprises:
identifying a plurality of cells on the WSI;
extracting a plurality of cellular features of different types for an individual cell, whereby respective pluralities of cellular features of different types are obtained for the plurality of cells;
using the trained machine-learning model to process the plurality of cellular features of different types instead of the plurality of averaged cellular features to thereby generate the plurality of probabilities of having malignancy so as to compute an average probability of having malignancy over the plurality of ensembled networks as a malignancy prediction score for indicating a likelihood that the individual cell is a potential CA cell, whereby a plurality of malignancy prediction scores is respectively generated for the plurality of cells;
using a density estimation model to estimate a possible-CA cell density distribution over the WSI from the plurality of malignancy prediction scores;
comparing the possible-CA cell density distribution against a low-risk threshold and a high-risk threshold to identify low-risk and high-risk regions containing CA, respectively; and
generating the tumor probability heatmap according to the identified low-risk and high-risk regions.
18 . The method of claim 17 , wherein the density estimation model is selected from a group consisting of a Gaussian model, a tophat model, an Epanechnikov model, an exponential model, a linear model and a cosine model.