Machine learning dental segmentation system and methods using graph-based approaches
Provided herein are systems and methods for automatically segmenting a 3D model of a patient's teeth. A patient's dentition may be scanned. The scan data may be converted into a 3D model, including a graph-based representation of the 3D model. The graph-based representation can be input into a machine learning model to train the machine learning model to segment the 3D model into individual dental components. Trained machine learning models can also be used to segment graph-based representations of a 3D model of a patient's teeth.
1 . A method of training a machine learning model to segment a 3D dental model, the method comprising:
receiving, in a computing device, a three-dimensional (3D) mesh of a patient's dentition;
extracting mesh features from the 3D mesh, wherein the mesh features include face adjacency matrices describing adjacent and non-adjacent faces with respect to the 3D mesh;
creating a graph-based representation of the 3D mesh with the extracted mesh features wherein the face adjacency matrices of the extracted mesh features are edges of the graph-based representation; and
applying the mesh features and/or the graph-based representation to train a machine learning model of the computing device to recognize graph-based segmentation elements corresponding to segmentation of the patient's dentition.
2 . The method of claim 1 , wherein the mesh features further include mesh faces of the 3D mesh.
3 . The method of claim 1 , wherein creating the graph-based representation further comprises using mesh faces as nodes of the graph-based representation.
4 . The method of claim 3 , further comprising storing the graph-based representation in a memory of the computing device.
5 . The method of claim 1 , wherein the mesh features can be one or more of a face center position, a face normal vector, a local curvature, and a face area.
6 . The method of claim 1 , further comprising segmenting the graph-based representation into individual dental components with the machine learning model.
7 . The method of claim 1 , wherein the 3D mesh comprises a scan of the patient's dentition.
8 . The method of claim 1 , further comprising repeating the receiving, extracting, creating, and applying steps for a plurality of 3D meshes of patients' dentitions.
9 . The method of claim 1 , wherein the machine learning model is trained to construct a semantic segmentation network in which multiple objects of the same class are treated as a single entity.
10 . The method of claim 1 , wherein the machine learning model is trained to construct an instance segmentation network in which multiple objects of the same class are treated as distinct individual objects or instances.
11 . A method of training a machine learning model to segment a 3D dental model, the method comprising:
receiving, in a computing device, a three-dimensional (3D) mesh of a patient's dentition;
creating, in the computing device, a graph-based representation of the 3 D mesh that represents the patient's dentition;
receiving, in the computing device, a ground truth input comprising a manual segmentation of the 3D mesh, wherein the ground truth input identifies individual teeth, gingiva, and interproximal spaces; and
training a machine learning model of the computing device to produce a segmentation output that attempts to achieve the ground truth input.
12 . The method of claim 11 , further comprising segmenting the graph-based representation into individual dental components with the trained machine learning model.
13 . The method of claim 11 , further comprising identifying interproximal spaces between teeth with the trained machine learning model.
14 . The method of claim 11 , further comprising identifying individual teeth with the trained machine learning model.
15 . The method of claim 11 , further comprising repeating the receiving, creating, receiving, and training steps for a plurality of 3D meshes of patients' dentitions.
16 . The method of claim 11 , wherein training the machine learning model comprises adjusting weights of the machine learning model to minimize an error between the ground truth input and the segmentation output.
17 . A method of segmenting a 3D dental model, the method comprising:
receiving, in a computing device, a three-dimensional (3D) mesh of a patient's dentition;
creating a graph-based representation of the 3D mesh;
applying the graph-based representation to a trained machine learning model of the computing device to recognize segmentation elements corresponding to segmentation of the patient's dentition, wherein the graph-based representation includes mesh features with face adjacency matrices describing adjacent and non-adjacent faces with respect to the 3D mesh as edges of the graph-based representation; and
outputting a segmented 3D model of the patient's dentition.
18 . A method of training a machine learning model to segment a 3D dental model, the method comprising:
receiving, in a computing device, a three-dimensional (3D) mesh of a patient's dentition;
extracting mesh features from the 3D mesh, wherein the mesh features include face adjacency matrices describing adjacent and non-adjacent faces with respect to the 3D mesh;
coarsening the 3D mesh to reduce a size of the mesh;
creating a graph-based representation of the coarsened 3D mesh with the extracted mesh features using face adjacency matrices as edges of the graph-based representation; and
applying the mesh features and/or the graph-based representation to train a machine learning model of the computing device to recognize graph-based segmentation elements corresponding to segmentation of the patient's dentition.
19 . The method of claim 18 , wherein coarsening the 3D mesh comprises:
computing a cosine similarity of the mesh features;
eliminating mesh features with a cosine similarity below a first threshold; and
applying pooling to remaining mesh features.
20 . The method of claim 19 , wherein the pooling comprises Graclus pooling.
21 . The method of claim 19 , wherein the pooling is repeated until a number of remaining nodes in the 3D mesh is less than or equal to a second threshold.
22 . The method of claim 19 , wherein the first threshold comprises 0.995.
23 . The method of claim 21 , wherein the second threshold comprises 200,000.