Systems and methods for review of computer-aided detection of pathology in images
Disclosed and described herein are systems and methods of performing computer-aided detection (CAD)/diagnosis (CADx) in medical images and comparing the results of the comparison. Such detection can be used for treatment plans and verification of claims produced by healthcare providers, for the purpose of identifying discrepancies between the two. In particular, embodiments disclosed herein are applied to identifying dental caries (“caries”) in radiographs and comparing them against progress notes, treatment plans, and insurance claims.
1. A system for performing computer-aided detection (CAD) using an image of one or more teeth, comprising:
a computer comprising one or more processors and a memory, wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer causing the computer to:
receive the image of the one or more teeth;
segment each of the one or more teeth that comprise the image to determine boundaries between each of the one or more teeth, wherein each tooth of the one or more teeth comprises a separate segmented image;
number each tooth in the separate segmented images using a tooth-type classifier model, wherein the teeth are numbered according to a standardized numbering system,
wherein numbering each tooth in the separate segmented images comprises:
performing a fuzzy-logic process for the numbering each tooth in the separate segmented images comprising:
identifying a likelihood that each of the separate segmented images contains any of several types of teeth including primary molars, secondary molars, primary canines, secondary canines, secondary premolars, primary incisors, secondary incisors, gaps between teeth, exfoliating teeth, and cropped teeth, wherein the probabilities of tooth types for the teeth are arranged in a tooth probability list of length number of teeth multiplied by a number of tooth types,
and wherein the tooth probability list is element-wise multiplied against each row of a matrix to produce a sequence score and tooth numbering is determined by the row with the highest score.
2. The system of claim 1 , wherein the standardized numbering system comprises a Universal Numbering System, wherein each tooth is assigned a number from 1-32 (adult teeth) or letter from A-T (deciduous teeth).
3. The system of claim 1 , wherein the image comprises an x-ray image of the one or more teeth, an image from a magnetic resonance imaging (MRI) system of the one or more teeth, an image from a computed tomography (CT) system of the one or more teeth, an image from an ultrasound system of the one or more teeth, an image from a positron emission tomography (PET) system of the one or more teeth, or an image from a single-photon emission computed tomography (SPECT) machine of the one or more teeth.
4. The system of claim 1 , wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to:
perform computer-aided detection (CAD) on each of the separate segmented images using a classifier model for restoration detection, wherein the restoration detection identifies teeth with restorations that may produce false negative results from a caries classifier model.
5. The system of claim 4 , wherein the classifier model classifies overall image features, said classifier model comprising an image classifier that is trained to identify images that include orthodontia or images that are of poor quality.
6. The system of claim 5 , wherein the classifier model comprises a convolutional neural network (CNN).
7. The system of claim 4 , wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to:
exclude separate segmented images with restorations that may produce false negative results from the caries classifier model; and
determine whether a tooth in a non-excluded separate segmented image has a caries, or not, using the caries classifier model, wherein the caries classifier is trained to determine whether a tooth has caries.
8. The system of claim 7 , wherein the caries classifier model comprises a convolutional neural network (CNN).
9. The system of claim 8 , wherein the caries classifier model comprises a region-based convolutional neural network (R-CNN).
10. The system of claim 1 , wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to execute a caries classifier model, said caries classifier model comprising a region-based convolutional neural network (R-CNN), and wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further cause the computer to perform caries detection of the image by the caries classifier model returning a list or an image of rectangular regions of the image with caries.
11. The system of claim 1 , wherein the one or more processors further execute computer-executable instructions to:
segment each of the one or more teeth that comprise the image to separate segmented images by separating upper teeth from lower teeth in the image by identifying an occlusal plane, wherein the occlusal plane is identified by rotating the image several degrees in clockwise and counterclockwise directions, where at each increment a projection of intensity for each image row is summed and the occlusal plane is identified by a valley in the profiles of this projection (“occlusal valley”);
examine each column in the rotated image, identify a maximum depth of the occlusal valley in that column, and create an occlusal curve, wherein the occlusal curve is a smoothed curve of the maximum valley depth locations spanned along the rows of the image;
identify boundaries between each tooth or partial tooth that comprises the image, wherein the boundaries are determined independently for an upper and lower set of teeth as determined by the occlusal plane, and wherein the boundaries between each tooth are determined by taking a column-wise projection of intensities between the occlusal curve and an edge of the image, wherein spaces between the teeth are indicated by the valleys in the projections; and
use K-means to determine an average position of column numbers in the image that correspond to the interproximal space between teeth.
12. The system of claim 11 , wherein the one or more processors further execute computer-executable instructions to:
starting at a row on the occlusal curve that roughly separates a tooth from one next to it, move a rectangular window towards the edge of the image, wherein a tooth segmentation line is chosen to be at a column of this window for which the average intensity in the window is the minimum;
filter out false tooth segmentation curves in the column-wise projections by comparing image intensity along the curve to those of curves translated to the left and right of the segmentation curve, wherein segmentation curves that run through the pulp of a tooth will have more similar intensities to each other, compared to a segmentation curve that runs between the teeth;
determine a distance between intensities, wherein curves for which the intensities are too close to each other are rejected; and
output separate segmented images, one for each segmented region.
13. The system of claim 1 , wherein the processor further executes computer-executable instructions to place the number assigned to each of the one or more teeth by the tooth-type classifier model in each tooth's separate segmented image.
14. The system of claim 1 , wherein the processor further executes computer-executable instructions to place the number assigned to each of the one or more teeth by the tooth-type classifier model in the image of the one or more teeth.
15. A method for performing computer-aided detection (CAD) using an image of one or more teeth, said method comprising:
segmenting and numbering each of the one or more teeth of the image using a computer comprising one or more processors and a memory, wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer causing the computer to:
segment each of the one or more teeth that comprise the image to determine boundaries between each of the one or more teeth, wherein each tooth of the one or more teeth comprises a separate segmented image; and
number each tooth in the separate segmented images using a tooth-type classifier model, wherein the teeth are numbered according to a standardized numbering system,
wherein numbering each tooth in the separate segmented images comprises:
performing a fuzzy-logic process for the numbering of each tooth in the separate segmented images comprising:
identifying a probability that each of the separate segmented images contains any of a plurality of tooth types including primary molars, secondary molars, primary canines, secondary canines, secondary premolars, primary incisors, secondary incisors, gaps between teeth, exfoliating teeth, and cropped teeth, wherein the probabilities of tooth types for the teeth are arranged in a tooth probability list of length number of teeth multiplied by a number of the tooth types,
and wherein the tooth probability list is element-wise multiplied against each row of a matrix to produce a sequence score and tooth numbering is determined by the row with a highest score.
16. The method of claim 15 , wherein the standardized numbering system comprises a Universal Numbering System, wherein each tooth is assigned a number from 1-32 (adult teeth) or letter from A-T (deciduous teeth).
17. The method of claim 15 , wherein the image comprises an x-ray image of the one or more teeth, an image from a magnetic resonance imaging (MRI) system of the one or more teeth, an image from a computed tomography (CT) system of the one or more teeth, an image from an ultrasound system of the one or more teeth, an image from a positron emission tomography (PET) system of the one or more teeth, or an image from a single-photon emission computed tomography (SPECT) machine of the one or more teeth.
18. The method of claim 15 , wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to:
perform computer-aided detection (CAD) on each of the separate segmented images using a classifier model for restoration detection, wherein the restoration detection identifies teeth with restorations that may produce false negative results from a caries classifier model.
19. The method of claim 18 , wherein the classifier model classifies overall image features, said classifier model comprising an image classifier that is trained to identify images that include orthodontia or images that are of poor quality.
20. The method of claim 19 , wherein the classifier model comprises a convolutional neural network (CNN).
21. The method of claim 18 , wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to:
exclude separate segmented images with restorations that may produce false negative results from the caries classifier model; and
determine whether a tooth in a non-excluded separate segmented image has a caries, or not, using the caries classifier model, wherein the caries classifier is trained to determine whether a tooth has caries.
22. The method of claim 21 , wherein the caries classifier model comprises a convolutional neural network (CNN).
23. The method of claim 22 , wherein the caries classifier model comprises a region-based convolutional neural network (R-CNN).
24. The method of claim 15 , wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to execute a caries classifier model, said caries classifier model comprising a region-based convolutional neural network (R-CNN), and wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer further causing the computer to perform caries detection of the image by the caries classifier model returning a list or an image of rectangular regions of the image with caries.
25. The method of claim 15 , wherein the one or more processors further execute computer-executable instructions to:
segment each of the one or more teeth that comprise the image to separate segmented images by separating upper teeth from lower teeth in the image by identifying an occlusal plane, wherein the occlusal plane is identified by rotating the image several degrees in clockwise and counterclockwise directions, where at each increment a projection of intensity for each image row is summed and the occlusal plane is identified by a valley in the profiles of this projection (“occlusal valley”);
examine each column in the rotated image, identify a maximum depth of the occlusal valley in that column, and create an occlusal curve, wherein the occlusal curve is a smoothed curve of the maximum valley depth locations spanned along the rows of the image;
identify boundaries between each tooth or partial tooth that comprises the image, wherein the boundaries are determined independently for an upper and lower set of teeth as determined by the occlusal plane, and wherein the boundaries between each tooth are determined by taking a column-wise projection of intensities between the occlusal curve and an edge of the image, wherein spaces between the teeth are indicated by the valleys in the projections; and
use K-means to determine an average position of column numbers in the image that correspond to the interproximal space between teeth.
26. The method of claim 25 , wherein the one or more processors further execute computer-executable instructions to:
starting at a row on the occlusal curve that roughly separates a tooth from one next to it, move a rectangular window towards the edge of the image, wherein a tooth segmentation line is chosen to be at a column of this window for which the average intensity in the window is the minimum;
filter out false tooth segmentation curves in the column-wise projections by comparing image intensity along the curve to those of curves translated to the left and right of the segmentation curve, wherein segmentation curves that run through the pulp of a tooth will have more similar intensities to each other, compared to a segmentation curve that runs between the teeth;
determine a distance between intensities, wherein curves for which the intensities are too close to each other are rejected; and
output separate segmented images, one for each segmented region.
27. The method of claim 15 , wherein the processor further executes computer-executable instructions to place the number assigned to each of the one or more teeth by the tooth-type classifier model in each tooth's separate segmented image.
28. The method of claim 15 , wherein the processor further executes computer-executable instructions to place the number assigned to each of the one or more teeth by the tooth-type classifier model in the image of the one or more teeth.
29. A system for performing computer-aided detection (CAD) using an image of one or more teeth, comprising:
a computer comprising one or more processors and a memory, wherein the one or more processors of the computer execute computer-executable instructions stored in the memory of the computer causing the computer to:
segment each of the one or more teeth that comprise the image to form a separate segmented image for each tooth of the one or more teeth that comprise the image; and
number each tooth of the separate segmented images using a tooth-type classifier model, wherein the teeth are numbered according to a standardized numbering system,
wherein numbering each tooth of the separate segmented images comprises the tooth-type classifier identifying a probability that each of the separate segmented images contains any of a plurality of tooth types, wherein the probabilities of tooth types for the teeth are arranged in a list of length number of teeth multiplied by a number of the plurality of tooth types to form a tooth probability list,
and wherein the tooth probability list is element-wise multiplied against each row of a matrix to produce a sequence score and tooth numbering is determined by the row with a highest score.
30. A method for performing computer-aided detection (CAD) using an image of one or more teeth, said method comprising:
segmenting each of the one or more teeth that comprise the image to form a separate segmented image for each tooth of the one or more teeth that comprise the image; and
number each tooth in each of the separate segmented images using a tooth-type classifier model, wherein the teeth are numbered according to a standardized numbering system,
wherein numbering each tooth of the separate segmented images comprises the tooth-type classifier identifying a probability that each of the separate segmented images contains any of a plurality of tooth types, wherein the probabilities of tooth types for the teeth are arranged in a list of length number of teeth multiplied by a number of the plurality of tooth types to form a tooth probability list,
and wherein the tooth probability list is element-wise multiplied against each row of a matrix to produce a sequence score and tooth numbering is determined by the row with a highest score.