Deep learning for tooth detection and evaluation
A method includes receiving an image of a face of a patient, the image including a depiction of teeth; processing the image of the face using one or more trained machine learning model, wherein the one or more trained machine learning model outputs a pixel-level classification of pixels in the image, the pixel level classification comprising a first set of pixels classified as being inside of a bounding shape that bounds a first plurality of teeth depicted in the image and a second set of pixels classified as being outside of the first bounding shape; cropping the image of the face of the patient, wherein the cropped image comprises depictions of the first plurality of teeth; and performing one or more operations on the cropped image of the face of the patient.
1. A method comprising:
receiving a two-dimensional (2D) image of a face of a patient, the 2D image including a depiction of teeth of the patient;
processing the 2D image of the face using one or more trained machine learning model, wherein the one or more trained machine learning model outputs a pixel-level classification of pixels in the 2D image, the pixel-level classification comprising a first set of pixels classified as being inside of a first bounding shape that bounds a first plurality of teeth of the teeth depicted in the 2D image of the face of the patient and a second set of pixels classified as being outside of the first bounding shape;
performing a classification comprising at least one of:
a) classifying, by the one or more trained machine learning model, the 2D image as depicting one of an anterior view, a side view or an occlusal view; or
b) classifying, by the one or more trained machine learning model, the 2D image as appropriate for further processing or as not appropriate for further processing;
cropping the 2D image of the face of the patient to leave a first region comprising the first set of pixels classified as being inside of the first bounding shape and to remove a second region comprising the second set of pixels classified as being outside of the first bounding shape, wherein the cropped 2D image comprises depictions of the first plurality of teeth; and
performing one or more operations on the cropped 2D image of the face of the patient.
2. The method of claim 1 , wherein the one or more trained machine learning model outputs a mask for the 2D image of the face of the patient, wherein each entry in the mask is associated with a specific pixel in the 2D image and indicates for the specific pixel whether the specific pixel is associated with the first region inside of the first bounding shape or the second region outside of the first bounding shape, wherein entries associated with pixels that are inside of the first bounding shape have a first value and wherein entries associated with pixels that are outside of the first bounding shape have a second value.
3. The method of claim 2 , wherein those pixels that are associated with entries in the mask having the second value are cropped.
4. The method of claim 1 , wherein the one or more operations comprise one or more image processing operations that comprise at least one of a tooth segmentation operation or an edge detection operation.
5. The method of claim 1 , wherein the one or more trained machine learning model comprises an artificial neural network.
6. The method of claim 1 , wherein the pixel-level classification of pixels in the 2D image that is output by the one or more trained machine learning model comprises a third set of pixels classified as being inside of a second bounding shape that bounds a second plurality of teeth of the teeth depicted in the 2D image of the face of the patient.
7. The method of claim 1 , wherein the 2D image is a 2D color image.
8. The method of claim 1 , further comprising classifying, by the one or more trained machine learning model, the first bounding shape as one of:
a bounding shape around left or right posterior teeth in an occlusal view;
a bounding shape around anterior teeth in an occlusal view;
a bounding shape around all teeth in a full jaw view;
a bounding shape around all teeth in an anterior view; or
a bounding teeth around all teeth in a left or right side view.
9. The method of claim 1 , wherein the first bounding shape bounds all of the teeth of the patient that are depicted in the 2D image.
10. A non-transitory computer readable medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving two-dimensional (2D) image of a face of a patient, the 2D image including a depiction of teeth of the patient;
processing the 2D image of the face using one or more trained machine learning model, wherein the one or more trained machine learning model outputs a pixel-level classification of pixels in the 2D image, the pixel-level classification comprising a first set of pixels classified as being inside of a first bounding shape that bounds a first plurality of teeth of the teeth depicted in the 2D image of the face of the patient and a second set of pixels classified as being outside of the first bounding shape;
performing a classification comprising at least one of:
a) classifying, by the one or more trained machine learning model, the 2D image as depicting one of an anterior view, a side view or an occlusal view; or
b) classifying, by the one or more trained machine learning model, the 2D image as appropriate for further processing or as not appropriate for further processing;
cropping the 2D image of the face of the patient to leave a first region comprising the first set of pixels classified as being inside of the first bounding shape and to remove a second region comprising the second set of pixels classified as being outside of the first bounding shape, wherein the cropped 2D image comprises depictions of the first plurality of teeth; and
performing one or more operations on the cropped 2D image of the face of the patient.
11. The non-transitory computer readable medium of claim 10 , wherein the one or more trained machine learning model outputs a mask for the 2D image of the face of the patient, wherein each entry in the mask is associated with a specific pixel in the 2D image and indicates for the specific pixel whether the specific pixel is associated with the first region inside of the first bounding shape or the second region outside of the first bounding shape, wherein entries associated with pixels that are inside of the first bounding shape have a first value and wherein entries associated with pixels that are outside of the first bounding shape have a second value.
12. The non-transitory computer readable medium of claim 11 , wherein those pixels that are associated with entries in the mask having the second value are cropped.
13. The non-transitory computer readable medium of claim 10 , wherein the one or more operations comprise one or more image processing operations that comprise at least one of a tooth segmentation operation or an edge detection operation.
14. The non-transitory computer readable medium of claim 10 , wherein the one or more trained machine learning model comprises an artificial neural network.
15. The non-transitory computer readable medium of claim 10 , wherein the pixel-level classification of pixels in the 2D image that is output by the one or more trained machine learning model comprises a third set of pixels classified as being inside of a second bounding shape that bounds a second plurality of teeth of the teeth depicted in the 2D image of the face of the patient.
16. The non-transitory computer readable medium of claim 10 , wherein the 2D image is a 2D color image.
17. The non-transitory computer readable medium of claim 10 , the operations further comprising classifying, by the one or more trained machine learning model, the first bounding shape as one of:
a bounding shape around left or right posterior teeth in an occlusal view;
a bounding shape around anterior teeth in an occlusal view;
a bounding shape around all teeth in a full jaw view;
a bounding shape around all teeth in an anterior view; or
a bounding teeth around all teeth in a left or right side view.
18. The non-transitory computer readable medium of claim 10 , wherein the first bounding shape bounds all of the teeth of the patient that are depicted in the 2D image.
19. The method of claim 1 , wherein the classification comprises:
classifying, by the one or more trained machine learning model, the 2D image as depicting one of the anterior view, the side view or the occlusal view.
20. The method of claim 1 , wherein the classification comprises:
classifying, by the one or more trained machine learning model, the 2D image as appropriate for further processing or as not appropriate for further processing.