Predicting DCIS recurrence risk using a machine learning-based high-content image analysis approach

1. A computer-implemented method for determining treatment of a patient, the method comprising:

receiving an image of living tissue of a patient, the image comprising a stained histopathology slide;

annotating, via a whole slide annotation routine, the entire image into tissue structures, wherein the whole slide annotation routine includes at least one preprocessing routing to preprocess the stained histopathology slide using whole slide color normalization and down-sampling, and at least one patch extraction routine, wherein the at least one patch extraction routine comprises using a sliding window to extract patches of the preprocessed stained histopathology slide;

extracting texture features from the annotated image;

determining a distribution of the extracted texture features relative to tissue conditions, wherein the determining step comprises quantifying the distribution of the extracted texture features within the annotated entire image and quantifying spatial relationships relative to the tissue conditions of the tissue structures;

classifying the patient into a risk group based on the determining step; and

treating the patient accordingly based on the risk group.

2. The method of claim 1 , wherein the annotating comprises:

color deconvoluting each of the plurality of patches to a plurality of stain layers;

extracting a plurality of texture features from the plurality of patches;

inputting the extracted texture features into a random forest to output a probability of each patch belonging to a category of tissue structure; and

combining the patch probabilities to form an image annotation of tissue structures.

3. The method of claim 2 , wherein the plurality of texture features comprise at least one selected texture feature, at least one convolutional neural network fully connected terminal layer features, or a combination of the two.

4. The method of claim 3 , wherein the determining the distribution comprises determining feature distributions, spatial features which compare distances between different tissue regions, and region confidence.

5. The method of claim 4 , wherein the classifying comprises:

selecting a plurality of features; and

inputting the selected features into at least one machine learning process to output a probability of a condition to be treated and a treatment recommendation.

6. The method of claim 1 , wherein the living tissue is breast tissue and the risk groups relate to risk of recurrence of breast cancer.

7. The method of claim 6 , wherein the breast cancer is ductal carcinoma in situ.

8. The method of claim 7 , wherein the categories of tissue structure comprise malignant duct, immune rich stroma, non-immune rich stroma, non-cancerous duct, and blood vessel.

9. A system for detecting malicious email messages, the system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

receiving an image of living tissue of a patient, the image comprising a stained histopathology slide;

annotating, via a whole slide annotation routine, the entire image into tissue structures, wherein the whole slide annotation routine includes at least one preprocessing routing to preprocess the stained histopathology slide using whole slide color normalization and down-sampling, and at least one patch extraction routine, wherein the at least one patch extraction routine comprises using a sliding window to extract patches of the preprocessed stained histopathology slide;

extracting texture features from the annotated image;

determining a distribution of the extracted texture features relative to tissue conditions, wherein the determining step comprises quantifying the distribution of the extracted texture features within the annotated entire image and quantifying spatial relationships relative to the tissue conditions of the tissue structures;

classifying the patient into a risk group based on the determining step; and

treating the patient accordingly based on the risk group.

10. The system of claim 9 , wherein the annotating comprises:

color deconvoluting each of the plurality of patches to a plurality of stain layers;

extracting a plurality of texture features from the plurality of patches;

inputting the extracted texture features into a random forest to output a probability of each patch belonging to a category of tissue structure; and

combining the patch probabilities to form an image annotation of tissue structures.

11. The system of claim 10 , wherein the plurality of texture features comprise at least one selected texture feature, at least one convolutional neural network fully connected terminal layer features, or a combination of the two.

12. The system of claim 11 , wherein the determining the distribution comprises determining feature distributions, spatial features which compare distances between different tissue regions, and region confidence.

13. The system of claim 12 , wherein the classifying comprises:

selecting a plurality of features; and

inputting the selected features into at least one machine learning process to output a probability of a condition to be treated and a treatment recommendation.

14. The system of claim 9 , wherein the living tissue is breast tissue and the risk groups relate to risk of recurrence of breast cancer.

15. The system of claim 14 , wherein the breast cancer is ductal carcinoma in situ.

16. The system of claim 15 , wherein the categories of tissue structure comprise malignant duct, immune rich stroma, non-immune rich stroma, non-cancerous duct, and blood vessel.

17. A computer program product for detecting malicious email messages, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

receiving an image of living tissue of a patient, the image comprising a stained histopathology slide;

annotating, via a whole slide annotation routine, the entire image into tissue structures, wherein the whole slide annotation routine includes at least one preprocessing routing to preprocess the stained histopathology slide using whole slide color normalization and down-sampling, and at least one patch extraction routine, wherein the at least one patch extraction routine comprises using a sliding window to extract patches of the preprocessed stained histopathology slide;

extracting texture features from the annotated image;

determining a distribution of the extracted texture features relative to tissue conditions, wherein the determining step comprises quantifying the distribution of the extracted texture features within the annotated entire image and quantifying spatial relationships relative to the tissue conditions of the tissue structures;

classifying the patient into a risk group based on the determining step; and

treating the patient accordingly based on the risk group.

18. The computer program product of claim 17 , wherein the annotating comprises:

color deconvoluting each of the plurality of patches to a plurality of stain layers;

extracting a plurality of texture features from the plurality of patches;

inputting the extracted texture features into a random forest to output a probability of each patch belonging to a category of tissue structure; and

combining the patch probabilities to form an image annotation of tissue structures.

19. The computer program product of claim 18 , wherein the plurality of texture features comprise at least one selected texture feature, at least one convolutional neural network fully connected terminal layer features, or a combination of the two.

20. The computer program product of claim 19 , wherein the determining the distribution comprises determining feature distributions, spatial features which compare distances between different tissue regions, and region confidence.

21. The computer program product of claim 20 , wherein the classifying comprises:

selecting a plurality of features; and

inputting the selected features into at least one machine learning process to output a probability of a condition to be treated and a treatment recommendation.

22. The computer program product of claim 17 , wherein the living tissue is breast tissue and the risk groups relate to risk of recurrence of breast cancer.

23. The computer program product of claim 22 , wherein the breast cancer is ductal carcinoma in situ.

24. The computer program product of claim 23 , wherein the categories of tissue structure comprise malignant duct, immune rich stroma, non-immune rich stroma, non-cancerous duct, and blood vessel.