Automated method for aligning three dimensional dental library model to three dimensional oral scan data and computer readable medium having program for performing the method
An automated method for aligning a 3D dental library model to 3D oral scan data includes determining a valid tooth of the 3D oral scan data, extracting scan landmarks of the 3D oral scan data, loading a dental library model corresponding to the valid tooth of the 3D oral scan data, extracting library landmarks of the 3D dental library model, initial-aligning the 3D dental library model to the 3D oral scan data using the scan landmarks and the library landmarks and matching an individual tooth of the 3D dental library model and an individual tooth of the 3D oral scan data.
1. An automated method for aligning a 3D dental library model to 3D oral scan data, the method comprising:
determining a valid tooth of the 3D oral scan data;
extracting scan landmarks of the 3D oral scan data;
loading the 3D dental library model corresponding to the valid tooth of the 3D oral scan data;
extracting library landmarks of the 3D dental library model;
initial-aligning the 3D dental library model to the 3D oral scan data using the scan landmarks and the library landmarks; and
matching an individual tooth of the 3D dental library model and an individual tooth of the 3D oral scan data,
wherein the extracting the scan landmarks of the 3D oral scan data comprises:
generating a 2D depth image based on the 3D oral scan data;
extracting 2D coordinates of the scan landmarks from the 2D depth image; and
inverse-projecting the 2D coordinates to the 3D oral scan data.
2. The method of claim 1 , further comprising segmenting teeth of the 3D oral scan data to generate teeth segmentation data.
3. The method of claim 1 , wherein the initial-aligning the 3D dental library model to the 3D oral scan data comprises:
matching a second arch formed by teeth of the 3D dental library model with a first arch formed by teeth of the 3D oral scan data.
4. The method of claim 1 , wherein the scan landmarks include at least three landmarks disposed in the 3D oral scan data, and
wherein the library landmarks include at least three landmarks disposed in the 3D dental library model.
5. The method of claim 4 , wherein the scan landmarks include a first landmark disposed at a first end of a first arch formed by teeth of the 3D oral scan data, a second landmark disposed at a second end of the first arch and a third landmark disposed at a central point of the first arch, and
wherein the library landmarks include a fourth landmark disposed at a first end of a second arch formed by teeth of the 3D dental library model, a fifth landmark disposed at a second end of the second arch and a sixth landmark disposed at a central point of the second arch.
6. The method of claim 4 , wherein the scan landmarks include a first landmark disposed at a last tooth in a first end in a horizontal direction of the 3D oral scan data, a second landmark disposed at a last tooth in a second end in the horizontal direction of the 3D oral scan data and a third landmark disposed at a center of two central incisors of the 3D oral scan data, and
wherein the library landmarks include a fourth landmark disposed at a last tooth in a first end in a horizontal direction of the 3D dental library model, a fifth landmark disposed at a last tooth in a second end in the horizontal direction of the 3D dental library model and a third landmark disposed at a center of two central incisors of the 3D dental library model.
7. The method of claim 1 , wherein the matching the individual tooth of the 3D dental library model and the individual tooth of the 3D oral scan data comprises:
matching a center of the individual tooth of the 3D dental library model with a center of the individual tooth of the 3D oral scan data.
8. The method of claim 7 , wherein the matching the individual tooth of the 3D dental library model and the individual tooth of the 3D oral scan data comprises:
rotating the individual tooth of the 3D dental library model and adjusting a size of the individual tooth of the 3D dental library model to minimize a difference between an angle of the individual tooth of the 3D dental library model and an angle of the individual tooth of the 3D oral scan data and a difference between the size of the individual tooth of the 3D dental library model and a size of the individual tooth of the 3D oral scan data.
9. The method of claim 1 , wherein a first neural network is used in the determining the valid tooth of the 3D oral scan data, and
wherein a second neural network different from the first neural network is used in the extracting the scan landmarks of the 3D oral scan data.
10. The method of claim 9 , wherein an input of the first neural network is the 3D oral scan data and an output of the first neural network is individual tooth information of the 3D oral scan data.
11. The method of claim 10 , wherein the individual tooth information is a scalar value or a labeling value expressed on the 3D oral scan data.
12. The method of claim 9 , wherein an input of the first neural network is the 3D oral scan data and an output of the first neural network is teeth segmentation data of the 3D oral scan data including segmented teeth of the 3D oral scan data.
13. The method of claim 9 , wherein an input of the second neural network is the 3D oral scan data and an output of the second neural network is 3D coordinates of the scan landmarks.
14. The method of claim 9 , wherein the second neural network is used in the extracting the library landmarks of the 3D dental library model.
15. A non-transitory computer-readable storage medium having stored thereon program instructions, which when executed by at least one hardware processor, performs the method of claim 1 .