Histomorphometric classifier to predict cardiac failure from whole-slide hematoxylin and eosin stained images

1. An apparatus for predicting heart failure, the apparatus comprising:

a processor;

a memory that stores a digital whole slide image (WSI) of a region of tissue derived from a heart transplant, an explant, a surgical tissue sample, or an endomyocardial biopsy, where the digital WSI has a plurality of pixels, and where a pixel has an intensity or a red-green-blue (RGB) color value;

an input/output (I/O) interface;

a set of circuits comprising a pre-processing circuit, an image acquisition circuit, a deep learning circuit, and a classification circuit; and

an interface to connect the processor, the memory, the I/O interface and the set of circuits:

where the pre-processing circuit is configured to generate a pre-processed WSI by downsampling the digital WSI;

where the image acquisition circuit is configured to randomly select a set of non-overlapping regions of interest (ROI)s from the pre-processed WSI, and configured to provide the set of non-overlapping ROIs to the deep learning circuit, where the set of non-overlapping ROIs has an odd cardinality;

where the deep learning circuit is configured to generate an image-level probability that a member of the set of non-overlapping ROIs is a failure/abnormal pathology ROI, where the deep learning circuit provides the member of the set of non-overlapping ROIs to a convolutional neural network (CNN), and where the CNN produces the image-level probability based, at least in part, on the member of the set of non-overlapping ROIs; and

where the classification circuit is configured to generate a patient-level probability that the patient from which the region of tissue represented in the WSI was acquired is experiencing failure or non-failure based, at least in part, on the image-level probability.

2. The apparatus of claim 1 , where the set of non-overlapping ROIs extracted by the image acquisition circuit includes eleven non-overlapping ROIs, and where a member of the set of non-overlapping ROIs has dimensions of 256 pixels by 256 pixels.

3. The apparatus of claim 1 , where the WSI is an hematoxylin and eosin (H&E) stained WSI acquired at a 20× magnification.

4. The apparatus of claim 3 , where the pre-processing circuit downsamples the WSI to an apparent magnification of 5×.

5. The apparatus of claim 1 , where the deep learning circuit includes a CNN configured to accept one-hundred 64 pixel by 64 pixel input patches per member of the set of non-overlapping ROIs.

6. The apparatus of claim 5 , where the CNN is a seven-layer CNN comprising:

a first layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, with a stride of 1;

a second layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 2, and a stride of 2;

a third layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 1;

a fourth layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 2;

a fifth layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 1;

a sixth layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 4, and a stride of 2; and

a fully connected layer having a 2 kernels of size 5.

7. The apparatus of claim 5 , where the classification circuit generates the patient-level probability based on a majority vote of image-level probabilities associated with members of the set of non-overlapping ROIs.

8. The apparatus of claim 7 , further comprising a training circuit configured to train the CNN, where training the CNN includes:

accessing a training dataset, where the training dataset includes a first subset of WSIs of tissue acquired from patients demonstrating clinically diagnosed heart failure, and a second, disjoint subset of WSIs of tissue acquired from patients that have not been clinically diagnosed with heart failure;

splitting the training dataset into k−1 groups, where k is an integer;

training the CNN with the first k−1 groups;

testing the CNN with the remaining group; and

upon determining that all patients have been used for training the CNN and testing the CNN:

ending the training.

9. A non-transitory computer-readable storage device storing computer-executable instructions that when executed by a computer controls the computer to perform a method for predicting cardiac failure, the method comprising:

accessing a digitized whole slide image (WSI) of cardiovascular histopathology;

generating a pre-processed WSI by downsampling the WSI;

extracting a set of non-overlapping regions of interest (ROIs) from the pre-processed WSI, where the set of non-overlapping ROIs has an odd cardinality;

providing the set of non-overlapping ROIs to a deep learning convolutional neural network (CNN);

receiving, from the CNN, a probability that a member of the set of non-overlapping ROIs a failure/abnormal pathology ROI; and

controlling a computer assisted diagnosis (CADx) system to classify the region of tissue represented in the WSI as a non-failure histopathology or as a failure/abnormal pathology histopathology based, at least in part, on the probabilities associated with members of the set of non-overlapping ROIs.

10. The non-transitory computer-readable storage device of claim 9 , where downsampling the digitized WSI includes downsampling the WSI to an apparent magnification of 5×.

11. The non-transitory computer-readable storage device of claim 9 , where a member of the set of non-overlapping ROIs has dimensions of 256 pixels by 256 pixels.

12. The non-transitory computer-readable storage device of claim 9 , where the set of non-overlapping ROIs is selected randomly from the WSI.

13. The non-transitory computer-readable storage device of claim 12 , where the set of non-overlapping ROIs includes 11 non-overlapping ROIs.

14. The non-transitory computer-readable storage device of claim 9 , where the digitized WSI represents hematoxylin and eosin (H&E) stained tissue derived from a transplant, an explant, a surgical tissue sample, or an endomyocardial biopsy.

15. The non-transitory computer-readable storage device of claim 14 , where the CADx system classifies the region of tissue based on a majority vote of the probabilities associated with members of the set of non-overlapping ROIs.

16. The non-transitory computer-readable storage device of claim 14 , the method further comprising training the CNN, where training the CNN includes:

accessing a training dataset, where the training dataset includes a first subset of WSIs of tissue acquired from patients demonstrating clinically diagnosed heart failure, and a second, disjoint subset of WSIs of tissue acquired from patients that have not been clinically diagnosed with heart failure;

splitting the training dataset into k−1 groups, where k is an integer;

training the CNN with the first k−1 groups;

testing the CNN with the remaining group;

upon determining that all patients represented in the training dataset have been used for training the CNN and testing the CNN:

ending the training.

17. The non-transitory computer-readable storage device of claim 9 , where the CNN accepts one-hundred 64 pixel by 64 pixel input patches per member of the set of non-overlapping ROIs.

18. The non-transitory computer-readable storage device of claim 17 , where the CNN is a seven-layer CNN.

19. The non-transitory computer-readable storage device of claim 18 , where the CNN comprises:

a first layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 1;

a second layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 2, and a stride of 2;

a third layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 1;

a fourth layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 2;

a fifth layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 3, and a stride of 1;

a sixth layer comprising a convolutional layer, a batch normalization layer, and an activation layer, the convolutional layer having 16 kernels of size 4, and a stride of 2; and

a fully connected layer having 2 kernels of size 5.

20. A computer-readable storage device storing computer-executable instructions that, in response to execution, cause a computer assisted diagnosis (CADx) system to perform operations comprising:

accessing a digitized whole slide image (WSI) of cardiovascular histopathology acquired at 20× magnification;

generating a pre-processed digitized WSI by downsampling the digitized WSI to 5× apparent magnification;

extracting a set of eleven non-overlapping regions of interest (ROIs) from the pre-processed digitized WSI, where a member of the set of eleven non-overlapping ROIs has dimensions of 256 pixels by 256 pixels;

providing the set of eleven non-overlapping ROIs to an unsupervised deep learning convolutional neural network (CNN), where the CNN accepts one-hundred 64 pixel by 64 pixel patches from a member of the set of eleven non-overlapping ROIs, where the CNN is a seven-layer CNN having less than 14000 neurons, where the CNN is trained using three-fold cross validation using a training dataset of WSIs of left ventricular tissue acquired from at least 200 patients, and where the at least 200 patients include a first cohort diagnosed with end-stage heart failure, and a second, different cohort without heart failure, where a WSI associated with the first cohort is labeled as failure/abnormal, and a WSI associated with the second cohort is labeled as non-failure;

receiving, from the CNN, an image-level probability that a member of the set of eleven non-overlapping ROIs is a failure/abnormal pathology ROI; and

computing a patient-level probability that the cardiovascular histopathology represented in the digitized WSI is non-failure failure/abnormal histopathology based, at least in part, on a majority vote of the probabilities associated with members of the set of non-overlapping ROIs.