Granted Patent B2

US 8,379,961 · App. 12/496,785 · Granted Feb 19, 2013

Abstract

Claims (37)

1. A method for detecting and counting mitotic figures in an image of a biopsy sample stained with at least one dye, the method comprising the steps of:

color filtering the image in a computer process to identify pixels in the image that have a color which is indicative of a mitotic figure;

extracting the mitotic pixels in the image that are connected to one another in a computer process, thereby producing blobs of mitotic pixels;

shape-filtering the blobs of mitotic pixels in a computer process to produce mitotic figure candidates;

clustering neighboring ones of the candidates in a computer process to produce refined mitotic figure candidates;

extracting sub-images of the refined mitotic figure candidates in a computer process by cropping the biopsy sample image at the location of the blobs;

extracting two sets of features from the sub-images of the refined mitotic figure candidates in two separate computer processes;

determining which of the mitotic figure candidates are mitotic figures in a computer classification process based on the extracted sets of features; and

counting the number of mitotic figures per square unit of biopsy sample tissue.

2. The method of claim 1 , wherein the color filtering step comprises the steps of:

extracting color histograms from the image to produce image color histograms.

3. The method of claim 2 , wherein the color filtering step further comprises the steps of:

predicting color thresholds from the image color histograms using support vector regression (SVR).

4. The method of claim 3 , wherein the color filtering step further comprises the steps of:

using the color thresholds to select the pixels of the image that have the color which is indicative of a mitotic figure.

5. The method of claim 1 , wherein one of the two feature extraction computer processes comprises a convolutional neural network (CNN) computer process.

6. The method of claim 5 , wherein the feature extraction step comprises the step of applying the CNN computer process to sub-images of the mitotic figure candidate to obtain CNN features of the mitotic figure candidate.

7. The method of claim 1 , wherein one of the two feature extraction computer processes comprises the step of extracting heuristic features from the mitotic figure candidate blobs and sub-images.

8. The method of claim 7 , wherein the feature extraction step comprises the step of extracting one or more features from a contour of the mitotic candidate blob, the one or more features including curvature histogram, center of mass radii histogram and spectrum, blob mass, contour length, contour symmetry from center of mass, contour concavity.

9. The method of claim 8 , wherein the feature extraction step further comprises the step of extracting one or more heuristic features from the sub-image of the mitotic candidate, the one or more heuristic feature including histogram of cytoplasm colors, histogram of mitotic colors, presence of chromosomal bristles, and measures of the granularity of the image (roughness of texture).

10. The method of claim 1 , wherein the computer classification process comprises a support vector machine (SVM) classifier.

11. A system for detecting and counting mitotic figures in an image of a biopsy sample stained with at least one dye, the system comprising:

a processor executing instructions for:

color filtering the image to identify pixels in the image that have a color which is indicative of a mitotic figure;

extracting the mitotic pixels in the image that are connected to one another, thereby producing blobs of mitotic pixels;

shape-filtering the blobs of mitotic pixels to produce mitotic figure candidates;

clustering neighboring ones of the candidates to produce refined mitotic figure candidates;

extracting sub-images of the refined mitotic figure candidates by cropping the biopsy sample image at the location of the blobs;

extracting two sets of features from the sub-images of the refined mitotic figure candidates;

determining which of the mitotic figure candidates are mitotic figures based on the extracted sets of features; and

counting the number of mitotic figures per square unit of biopsy sample tissue.

12. The system of claim 11 , wherein one of the feature extraction instructions comprises a convolutional neural network (CNN) for identifying CNN features from the mitotic figure candidate sub-images.

13. The system of claim of 11 , wherein one of the feature extraction instructions comprises heuristic feature extractors for extracting heuristic features from the mitotic figure candidates' blob and sub-image.

14. The system of claim 13 , wherein the feature extraction instructions further comprises extractors for extracting one or more features from a contour of the mitotic candidate blob, the one or more features including curvature histogram, center of mass radii histogram and spectrum, blob mass, contour length, contour symmetry from center of mass, contour concavity.

15. The system of claim 13 , wherein the feature extraction instructions further comprises extractors for extracting one or more heuristic features from the sub-image of the mitotic candidate, the one or more heuristic features including histogram of cytoplasm colors, histogram of mitotic colors, presence of chromosomal bristles, and measure of granularity of the image (roughness of texture).

16. The system of claim 11 , wherein the instructions for determining which of the mitotic figure candidates are mitotic figures based on the extracted sets of features is performed comprises a support vector machine (SVM) classifier.

17. The system of claim 11 , wherein the instructions for counting the number of mitotic figures per square unit of biopsy sample tissue comprises counting the number of positively classified mitotic figures per square unit of biopsy sample tissue.

Assignments (6)