Scalable and high precision context-guided segmentation of histological structures including ducts/glands and lumen, cluster of ducts/glands, and individual nuclei in whole slide images of tissue samples from spatial multi-parameter cellular and sub-cellular imaging platforms
A method (and system) of segmenting one or more histological structures in a tissue image represented by multi-parameter cellular and sub-cellular imaging data includes receiving coarsest level image data for the tissue image, wherein the coarsest level image data corresponds to a coarsest level of a multiscale representation of first data corresponding to the multi-parameter cellular and sub-cellular imaging data. The method further includes breaking the coarsest level image data into a plurality of non-overlapping superpixels, assigning each superpixel a probability of belonging to the one or more histological structures using a number of pre-trained machine learning algorithms to create a probability map, extracting an estimate of a boundary for the: one or more histological structures by applying a contour algorithm to the probability map, and using the estimate of the boundary to generate a refined boundary for the one or more histological structures.
1. A method of segmenting one or more histological structures in a tissue image represented by multi-parameter cellular and sub-cellular imaging data, the method comprising:
receiving coarsest level image data for the tissue image, wherein the coarsest level image data corresponds to a coarsest level of a multiscale representation of first data corresponding to the multi-parameter cellular and sub-cellular imaging data;
breaking the coarsest level image data into a plurality of non-overlapping superpixels;
assigning each superpixel a probability of belonging to the one or more histological structures using a number of pre-trained machine learning algorithms to create a probability map;
extracting an estimate of a boundary for the one or more histological structures by applying a contour algorithm to the probability map; and
using the estimate of the boundary to generate a refined boundary for the one or more histological structures.
2. The method according to claim 1 , wherein the multiscale representation comprises a Gaussian multiscale pyramid decomposition, wherein the multi-parameter cellular and sub-cellular imaging data comprises stained tissue image data, wherein the receiving coarsest level image data for the tissue image comprises receiving coarsest level normalized constituent stain image data for the stained tissue image, wherein the coarsest level normalized constituent stain image data is for a particular constituent stain of the stained tissue image and corresponds to the coarsest level of the Gaussian multiscale pyramid decomposition of the first data corresponding to the stained tissue image data, and wherein the breaking the coarsest level image data into a plurality of superpixels comprises breaking the coarsest level normalized constituent stain image data into the plurality of superpixels.
3. The method according to claim 2 , wherein the multi-parameter cellular and sub-cellular imaging data comprises stained tissue image data, wherein the receiving coarsest level image data for the tissue image comprises receiving coarsest level normalized constituent stain image data for the stained tissue image, wherein the coarsest level normalized constituent stained tissue image data is generated by: (i) color deconvolving the stained tissue image data into respective stain intensities to create constituent stain image data, (ii) normalizing the constituent stain image data using a reference data set to create normalized data, and (iii) performing the Gaussian multiscale decomposition on the normalized data to produce a multiscale representation of the normalized data including the coarsest level normalized constituent stain image data, and wherein the first data corresponding to the stained tissue image data is the normalized data.
4. The method according to claim 2 , wherein the first data corresponding to the stained tissue image data is the stained tissue image data, wherein the coarsest level normalized constituent stain image data is generated by: (i) performing the Gaussian multiscale decomposition on the stained tissue image data to produce a multiscale representation of the stained tissue image data, (ii) color deconvolving the coarsest level of the Gaussian multiscale pyramid decomposition of the stained tissue image data into respective stain intensities to create coarsest level constituent stain image data, and (iii) normalizing the coarsest level constituent stain image data using a reference data set to create the coarsest level normalized constituent stain image data.
5. The method according to claim 2 , wherein the number of pre-trained machine learning algorithms are a number of supervised machine learning algorithms pre-trained based on user input.
6. The method according to claim 5 , wherein the number of pre-trained machine learning algorithms includes a context-SVM model or a context LR model and a stain-SVM model or a stain-LR model which are applied to the plurality of super pixels.
7. The method according to claim 2 , wherein the contour algorithm is a region-based active contour algorithm.
8. The method according to claim 2 , wherein the using the estimate of the boundary to generate a refined boundary for the one or more histological structures comprises up sampling boundaries of the one or more histological structures using levels of the Gaussian multiscale pyramid decomposition other than the coarsest level and utilizing a region-based active contour algorithm at a finest level of the Gaussian multiscale pyramid decomposition to create the refined boundary.
9. The method according to claim 2 , wherein the structure comprises a duct or gland, wherein the stained tissue image data comprises H&E data, and wherein the particular constituent stain is hematoxylin.
10. The method according to claim 2 , wherein the structure comprises connective tissue, wherein the stained tissue image data comprises H&E data, and wherein the particular constituent stain is eosin.
11. The method according to claim 1 , wherein the breaking the coarsest level image data into a plurality of non-overlapping superpixels employs a linear iterative clustering (SLIC) algorithm.
12. A non-transitory computer readable medium storing one or more programs, including instructions, which when executed by a computer, causes the computer to perform the method of claim 1 .
13. A computerized system for segmenting one or more histological structures in a tissue image represented by multi-parameter cellular and sub-cellular imaging data, comprising:
a processing apparatus, wherein the processing apparatus includes a number of components configured for:
receiving coarsest level image data for the tissue image, wherein the coarsest level image data corresponds to a coarsest level of a multiscale representation of first data corresponding to the multi-parameter cellular and sub-cellular imaging data;
breaking the coarsest level image data into a plurality of non-overlapping superpixels;
assigning each superpixel a probability of belonging to the one or more histological structures using a number of pre-trained machine learning algorithms to create a probability map;
extracting an estimate of a boundary for the one or more histological structures by applying a contour algorithm to the probability map; and
using the estimate of the boundary to generate a refined boundary for the one or more histological structures.
14. The system according to claim 13 , wherein the multiscale representation comprises a Gaussian multiscale pyramid decomposition, wherein the multi-parameter cellular and sub-cellular imaging data comprises stained tissue image data, wherein the receiving coarsest level image data for the tissue image comprises receiving coarsest level normalized constituent stain image data for the stained tissue image, wherein the coarsest level normalized constituent stain image data is for a particular constituent stain of the stained tissue image and corresponds to the coarsest level of the Gaussian multiscale pyramid decomposition of the first data corresponding to the stained tissue image data, and wherein the breaking the coarsest level image data into a plurality of superpixels comprises breaking the coarsest level normalized constituent stain image data into the plurality of superpixels.
15. The system according to claim 14 , wherein the multi-parameter cellular and sub-cellular imaging data comprises stained tissue image data, wherein the receiving coarsest level image data for the tissue image comprises receiving coarsest level normalized constituent stain image data for the stained tissue image, wherein the coarsest level normalized constituent stained tissue image data is generated by: (i) color deconvolving the stained tissue image data into respective stain intensities to create constituent stain image data, (ii) normalizing the constituent stain image data using a reference data set to create normalized data, and (iii) performing the Gaussian multiscale decomposition on the normalized data to produce a multiscale representation of the normalized data including the coarsest level normalized constituent stain image data, and wherein the first data corresponding to the stained tissue image data is the normalized data.
16. The system according to claim 14 , wherein the first data corresponding to the stained tissue image data is the stained tissue image data, wherein the coarsest level normalized constituent stain image data is generated by: (i) performing the Gaussian multiscale decomposition on the stained tissue image data to produce a multiscale representation of the stained tissue image data, (ii) color deconvolving the coarsest level of the Gaussian multiscale pyramid decomposition of the stained tissue image data into respective stain intensities to create coarsest level constituent stain image data, and (iii) normalizing the coarsest level constituent stain image data using a reference data set to create the coarsest level normalized constituent stain image data.
17. The system according to claim 14 , wherein the number of pre-trained machine learning algorithms are a number of supervised machine learning algorithms pre-trained based on user input.
18. The system according to claim 17 , wherein the number of pre-trained machine learning algorithms includes a context-SVM model or a context-LR model and a stain-SVM model or a stain-LR model which are applied to the plurality of super pixels.
19. The system according to claim 14 , wherein the contour algorithm is a region-based active contour algorithm.
20. The system according to claim 14 , wherein the using the estimate of the boundary to generate a refined boundary for the one or more histological structures comprises up sampling boundaries of the one or more histological structures using levels of the Gaussian multiscale pyramid decomposition other than the coarsest level and utilizing a region-based active contour algorithm at a finest level of the Gaussian multiscale pyramid decomposition to create the refined boundary.
21. The system according to claim 14 , wherein the structure comprises a duct or gland, wherein the stained tissue image data comprises H&E data, and wherein the particular constituent stain is hematoxylin.
22. The system according to claim 14 , wherein the structure comprises connective tissue, wherein the stained tissue image data comprises H&E data, and wherein the particular constituent stain is eosin.
23. The system according to claim 13 , wherein the breaking the coarsest level image data into a plurality of non-overlapping superpixels employs a linear iterative clustering (SLIC) algorithm.