Artificial intelligence architecture for identification of periodontal features
Dental images are processed according to a first machine learning model to determine teeth labels. The teeth labels and image are concatenated and processed using a second machine learning model to label anatomy including CEJ, JE, GM, and Bone. The anatomy labels, teeth labels, and image are concatenated and processed using a third machine learning model to obtain feature measurements, such as pocket depth and clinical attachment level. The feature measurements, anatomy labels, teeth labels, and image may be concatenated and input to a fourth machine learning model to obtain a diagnosis for a periodontal condition. Feature measurements and/or the diagnosis may be processed according to a diagnosis hierarchy to determine whether a treatment is appropriate. Machine learning models may further be used to reorient, decontaminate, and restore the image prior to processing. Machine learning models may be embodied as CNN, GAN, and cyclic GAN.
1. A method for processing a dental image comprising:
receiving, by a computer system, the dental image;
processing, by the computer system, the dental image using a tooth-labeling machine learning model to obtain one or more tooth labels of one or more teeth represented in the dental image;
concatenating, by the computer system, the dental image with the one or more tooth labels to obtain first concatenated data;
processing, by the computer system, the first concatenated data using an anatomy-labeling machine learning model to obtain one or more anatomy labels of anatomy represented in the dental image;
concatenating, by the computer system, all of the one or more tooth labels, the one or more anatomy labels, and the dental image to obtain second concatenated data;
processing, by the computer system, the second concatenated data using a measurement-labeling machine learning model to obtain one or more measurement values measuring one or more periodontal features of the one or more teeth represented in the dental image;
concatenating, by the computer system, all of the one or more tooth labels, the one or more anatomy labels, the one or more measurement values, and the dental image to obtain third concatenated data; and
processing, by the computer system, the third concatenated data using a diagnosis-labeling machine learning model to obtain a diagnosis of a periodontal disease of the one or more teeth represented in the dental image.
2. The method of claim 1 , wherein the one or more measurement values include any of pocket depth (PD) and clinical attachment level (CAL).
3. The method of claim 1 , further comprising:
evaluating, by the computer system, the one or more measurement values according to a diagnosis hierarchy; and
when one or more criteria of the diagnosis hierarchy are met, outputting, by the computer system, a response indicating that a course of treatment corresponding to the diagnosis hierarchy is appropriate.
4. The method of claim 1 , wherein the dental image is a pre-processed dental image, the method further comprising:
receiving, by the computer system, an original image;
processing, by the computer system, the original image according to an orientation-labeling machine learning model to obtain a first orientation of the original image; and
re-orienting, by the computer system, the original image from the first orientation to a standard orientation to obtain the pre-processed dental image.
5. The method of claim 1 , wherein the dental image is a pre-processed dental image, the method further comprising:
receiving, by the computer system, an original dental image; and
processing, by the computer system, the original dental image according to a contamination-removing machine learning model to obtain the pre-processed dental image.
6. The method of claim 5 , wherein the contamination-removing machine learning model is a generator trained using a cyclic generative adversarial network (GAN).
7. The method of claim 1 , wherein the dental image is a pre-processed dental image, the method further comprising:
receiving, by the computer system, an original dental image having missing portions; and
processing, by the computer system, the original dental image according to a restoring machine learning model to obtain the pre-processed dental image having information estimated for the missing portions.
8. The method of claim 1 , further comprising:
processing, by the computer system, the dental image according to view-classifying machine learning model to obtain a view classification, the view classification including at least one of an anatomic sequence identifier, an anatomic sequence modifier, and a laterality of anatomy represented in the dental image.
9. The method of claim 1 , wherein the tooth-labeling machine learning model and the anatomy-labeling machine learning model are each a convolution neural network.
10. The method of claim 1 , wherein the anatomy-labeling machine learning model comprises a plurality of machine learning models trained to identify, for at least one tooth label of the one or more tooth labels, at least two anatomical features from the group consisting of cementum-enamel junction (CEJ), junctional epithelium (JE), gingival margin, and bony points.
11. The method of claim 1 , wherein processing the first concatenated data using the anatomy-labeling machine learning model to obtain the one or more anatomy labels of the anatomy represented in the dental image comprises performing, by the computer system:
processing the dental image using a generator to obtain a probability distribution label corresponding to locations of pixels of the dental image corresponding to the anatomy;
concatenating the dental image, the one or more tooth labels, and the probability distribution label to obtain the first concatenated data; and
processing the first concatenated data using a classifier to obtain the one or more anatomy labels.
12. The method of claim 11 , wherein the generator and classifier are generated according to training cycles that includes repeatedly performing, using a computer system:
(a) providing a training data entry including a training image and including a tooth training label that labels pixels in the training image corresponding to one or more teeth and an anatomy training label that labels pixels corresponding to an anatomical feature represented in the training image;
(b) processing the tooth training label and the training image using the generator to obtain a synthetic label;
(c) processing the synthetic label and an unpaired label from a repository using a discriminator to obtain one or more realism values;
(d) concatenating the training image, tooth training label, and synthetic label to obtain training concatenated data;
(e) processing the training concatenated data using a classifier to obtain a training label estimate;
(e) evaluating a plurality of loss functions according to the one or more realism values and similarity of the training label estimate to the anatomy training label; and
(f) modifying the generator, discriminator, and classifier according to outputs of the plurality of loss functions.
13. The method of claim 12 , wherein (e) further comprises evaluating the plurality of loss functions with respect to similarity of a dilated version of the anatomy trying label and the synthetic label, the unpaired label being a dilated version of an anatomical label.
14. The method of claim 12 , wherein the generator is an encoder-decoder convolution neural network (CNN) and the discriminator and classifier are embodied as convolution neural networks.
15. A non-transitory computer-readable medium storing executable instructions that, when executed by a processing device, cause the processing device to:
receive a dental image;
process the dental image using a tooth-labeling machine learning model to obtain one or more tooth labels of one or more teeth represented in the dental image;
concatenate the dental image with the one or more tooth labels to obtain first concatenated data;
process the first concatenated data using an anatomy-labeling machine learning model to obtain one or more anatomy labels of anatomy represented in the dental image;
concatenate all of the one or more tooth labels, the one or more anatomy labels, and the dental image to obtain second concatenated data;
process the second concatenated data using a measurement-labeling machine learning model to obtain one or more measurement values measuring one or more periodontal features of the one or more teeth represented in the dental image;
concatenate all of the one or more tooth labels, the one or more anatomy labels, the one or more measurement values, and the dental image to obtain third concatenated data; and
process the third concatenated data using a diagnosis-labeling machine learning model to obtain a diagnosis of a periodontal disease of the one or more teeth represented in the dental image.
16. The non-transitory computer-readable medium of claim 15 , wherein the executable instructions, when executed by the processing device, further cause the processing device to:
evaluate the one or more measurement values according to a diagnosis hierarchy; and
when one or more criteria of the diagnosis hierarchy are met, output a response indicating that a course of treatment corresponding to the diagnosis hierarchy is appropriate.
17. The non-transitory computer-readable medium of claim 15 , wherein the one or more measurement values include any of pocket depth (PD) and clinical attachment level (CAL).
18. The non-transitory computer-readable medium of claim 15 , wherein the anatomy-labeling machine learning model comprises a plurality of machine learning models trained to identify, for at least one tooth label of the one or more tooth labels, at least two anatomical features from the group consisting of cementum-enamel junction (CEJ), junctional epithelium (JE), gingival margin, and bony points.