Apparatuses and methods for three-dimensional dental segmentation using dental image data
Methods and apparatuses (including systems and devices), including computer-implemented methods for segmenting, correcting and/or modifying a three-dimensional (3D) model of a subject's oral cavity to determine individual components such as teeth, gingiva, tongue, palate, etc., that may be selective and/or collectively digitally manipulated. In some implementations, artificial intelligence uses libraries of labeled 2D images and 3D dental models to learn how to segment a 3D dental model of a subject's oral cavity using 2D images, height map and/or other data and projection values that relate the 2D images to the 3D model. As noted herein, the dental classes can include a variety of intra-oral and extra-oral objects and can be represented as binary values, discrete values, a continuum of height map data, etc. In some implementations, several dental classes are predicted concurrently.
1. A computer-implemented method comprising:
identifying a plurality of two-dimensional (2D) images of a subject's oral cavity, wherein the plurality of 2D images each comprise height map data corresponding to a distance between an image capture device and at least a portion of the subject's oral cavity, and the plurality of 2D images are each associated with one or more projection values to relate the each 2D image to a digital three-dimensional (3D) model of the subject's oral cavity;
processing the plurality of 2D images to segment each 2D image of the plurality of 2D images into a plurality of dental classes; and
using the one or more projection values of each 2D image of the plurality of 2D images to project the segmented 2D images onto the 3D model to segment the 3D model into a segmented 3D model into the plurality of dental classes.
2. The computer-implemented method of claim 1 , further comprising collecting the plurality of 2D images and the height map data of each of the plurality of 2D images with an intraoral scanner.
3. The computer-implemented method of claim 1 , further comprising collecting the plurality of 2D images by identifying a view of the 3D model and generating a 2D projection of the 3D model from the view.
4. The computer-implemented method of claim 1 , further comprising collecting the plurality of 2D images and the height map data of each of the plurality of 2D images from scanned images of the subject's oral cavity.
5. The computer-implemented method of claim 1 , further comprising modifying the 2D images.
6. The computer-implemented method of claim 5 , wherein modifying comprises adjusting the height map data of each 2D image.
7. The computer-implemented method of claim 1 , wherein processing the plurality of 2D images comprises applying a trained machine-learning agent to segment each of the 2D images.
8. The computer-implemented method of claim 7 , wherein processing comprises using a conditional Generative Adversarial Network.
9. The computer-implemented method of claim 1 , wherein projecting the segmented 2D images onto the 3D model comprises resolving conflicts between the segmentation of each 2D image prior to projecting onto the 3D model.
10. The computer-implemented method of claim 9 , wherein resolving the conflicts comprises applying Bayes' Theorem, deconflicting the conflicts through voting, or some combination thereof.
11. The computer-implemented method of claim 1 , wherein the one or more projection values of each of the plurality of 2D images represents a projection of pixels on the each of the plurality of 2D images to a face or vertex of a mesh of the 3D model.
12. The computer-implemented method of claim 1 , wherein using the one or more projection values of each 2D image of the plurality of 2D images to project the segmented 2D images onto the 3D model comprises mapping one or more pixel values from pixels of the segmented 2D images onto one or more faces of a mesh of the 3D model.
13. The computer-implemented method of claim 1 , wherein the plurality of dental classes comprise teeth, gums, and excess materials.
14. A system comprising:
one or more processors;
a memory coupled to the one or more processors, the memory configured to store computer-program instructions, that, when executed by the one or more processors, perform a computer-implemented method comprising:
identifying a plurality of two-dimensional (2D) images of a subject's oral cavity, wherein the 2D images correspond to a digital three-dimensional (3D) model of the subject's oral cavity;
processing the plurality of 2D images to segment each 2D image into a plurality of different structures; and
projecting the segmented 2D images onto the 3D model to form a segmented 3D model.
15. The system of claim 14 , wherein the computer-implemented method further comprises: collecting the plurality of 2D images by identifying a view of the 3D model and generating a 2D projection of the 3D model from the view.
16. The system of claim 14 , wherein the computer-implemented method further comprises: collecting the plurality of 2D images from scanned images of the subject's oral cavity.
17. The system of claim 14 , wherein the computer-implemented method further comprises: modifying the 2D images.
18. The system of claim 17 , wherein the computer-implemented method further comprises modifying the 2D images by adjusting a height map of each 2D image.
19. The system of claim 14 , wherein the computer-implemented method further comprises: processing the plurality of 2D images by applying a trained machine-learning agent to segment each of the 2D images.
20. The system of claim 14 , wherein the computer-implemented method further comprises: processing using a conditional Generative Adversarial Network or other Neural Network.
21. The system of claim 14 , wherein the computer-implemented method comprises projecting the segmented 2D images onto the 3D model by resolving conflicts between the segmentation of each 2D image prior to projecting onto the 3D model.
22. The system of claim 21 , wherein the computer-implemented method further comprises: resolving the conflicts by applying Bayes' Theorem.