Inpainting dental images with missing anatomy
Dental images are processed according to a first machine learning model to determine teeth labels. The teeth labels and image are processed using a second machine learning model to label anatomy. The anatomy labels, teeth labels, and image are processed using a third machine learning model to obtain feature measurements, such as pocket depth and clinical attachment level. The feature measurements, labels, and image may be input to a fourth machine learning model to obtain a diagnosis for a periodontal condition. Machine learning models may further be used to reorient, decontaminate, and restore the image prior to processing. A machine learning model may be trained with images and randomly generated masks in order to perform inpainting of dental images with missing information.
1. A method for training a machine learning model comprising:
providing, on the computer system, a plurality of first dental images;
providing, on a computer system, a random mask for each image of the plurality of first dental images;
processing, by the computer system, each image of the plurality of first dental images and the random mask for the each image with a generator machine learning model to obtain an inpainted image for the each image, the inpainted image including image data generated by the generator machine learning model for image locations corresponding to the random mask for the each image; and
for each image of the plurality of first dental images, updating, by the computer system, the generator machine learning model according to a training algorithm according to similarity of the each image to the inpainted image obtained for the each image;
wherein each image of the plurality of first dental images is an image of dental anatomy according to an imaging modality selected from the group consisting of full mouth series X-rays, dental cone beam computed tomography (CBCT), cephalometric X-ray, intra-oral optical image, panoramic dental X-ray, dental magnetic resonance imaging (MRI) image, dental light detection and ranging (LIDAR) image.
2. The method of claim 1 , further comprising:
processing, by the computer system, the inpainted image for each first dental image of the plurality of first dental images and an unpaired dental image from a repository using a discriminator machine learning model to obtain a realism estimate; and
updating, by the training algorithm, the generator machine learning model and the discriminator machine learning model according to the realism estimates for the plurality of first dental images.
3. The method of claim 2 , wherein the generator machine learning model is an encoder-decoder machine learning model.
4. The method of claim 3 , wherein the generator machine learning model and the discriminator machine learning model are each convolution neural networks.
5. The method of claim 2 , wherein the realism estimate is a matrix of realism estimates.
6. The method of claim 1 , further comprising:
processing, by the computer system, a second dental image using the generator machine learning model to obtain a utilization inpainted image; and
processing, by the computer system, the utilization inpainted image according to a labeling machine learning model to obtain labels of dental anatomy in the utilization inpainted image.
7. The method of claim 6 , wherein processing the second dental image using the generator machine learning model comprises processing the second dental image in combination with a mask indicating missing portions of the second dental image.
8. The method of claim 6 , further comprising:
processing, by the computer system, the utilization inpainted image and labels of dental anatomy according to a measuring machine learning model to obtain a measurement of a dental condition.
9. The method of claim 8 , further comprising:
processing, by the computer system, the utilization inpainted image, labels of dental anatomy, and measurement of the dental condition according to a diagnosing machine learning model to obtain a diagnosis of a dental pathology.
10. The method of claim 9 , wherein the dental pathology is a diagnosis of a periodontal disease.
11. A non-transitory computer-readable medium storing executable instructions that, when executed by a processing device, cause the processing device to:
receive a plurality of first dental images;
receive or generate a random mask for each image of the plurality of first dental images;
process each image of the plurality of first dental images and the random mask for the each image with a generator machine learning model to obtain an inpainted image for the each image, the inpainted image including image data generated by the generator machine learning model for image locations corresponding to the random mask for the each image; and
for each image of the plurality of first dental images, update the generator machine learning model according to a training algorithm according to similarity of the each image to the inpainted image obtained for the each image;
wherein each image of the plurality of first dental images is an image of dental anatomy according to an imaging modality selected from the group consisting of full mouth series X-rays, dental cone beam computed tomography (CBCT), cephalometric X-ray, intra-oral optical image, panoramic dental X-ray, dental magnetic resonance imaging (MRI) image, dental light detection and ranging (LIDAR) image.
12. The non-transitory computer-readable medium of claim 11 , wherein the executable instructions, when executed by the processing device, further cause the processing device to:
process the inpainted image for each first dental image of the plurality of first dental images and an unpaired dental image from a repository using a discriminator machine learning model to obtain a realism estimate; and
update, by the training algorithm, the generator machine learning model and the discriminator machine learning model according to the realism estimates for the plurality of first dental images.
13. The non-transitory computer-readable medium of claim 12 , wherein the generator machine learning model is an encoder-decoder machine learning model.
14. The non-transitory computer-readable medium of claim 13 , wherein the generator machine learning model and the discriminator machine learning model are each convolution neural networks.
15. The non-transitory computer-readable medium of claim 12 , wherein the realism estimate is a matrix of realism estimates.
16. The non-transitory computer-readable medium of claim 11 , wherein the executable instructions, when executed by the processing device, further cause the processing device to:
process a second dental image using the generator machine learning model to obtain a utilization inpainted image; and
process the utilization inpainted image according to a labeling machine learning model to obtain labels of dental anatomy in the utilization inpainted image.
17. The non-transitory computer-readable medium of claim 16 , wherein the executable instructions, when executed by the processing device, further cause the processing device to process the second dental image using the generator machine learning model by processing the second dental image in combination with a mask indicating missing portions of the second dental image.
18. The non-transitory computer-readable medium of claim 16 , wherein the executable instructions, when executed by the processing device, further cause the processing device to:
process the utilization inpainted image and labels of dental anatomy according to a measuring machine learning model to obtain a measurement of a dental condition.
19. The non-transitory computer-readable medium of claim 18 , wherein the executable instructions, when executed by the processing device, further cause the processing device to:
process the utilization inpainted image, labels of dental anatomy, and measurement of the dental condition according to a diagnosing machine learning model to obtain a diagnosis of a dental pathology.
20. The non-transitory computer-readable medium of claim 19 , wherein the dental pathology is a diagnosis of a periodontal disease.