Providing a simulated outcome of dental treatment on a patient
Systems and methods of simulating dental treatments are disclosed. A method may include capturing a first 2D image of a patient's face, including the patient's teeth, building a parametric 3D model of the patient's teeth based on the 2D image, developing a simulated outcome of a dental treatment of the patient's teeth by rendering the 3D model with the patient's teeth in one or more positions and/or orientations corresponding to the treatment goals of the dental treatment plan, and rendering a second 2D image of the patient's face with teeth according to a simulated outcome of the dental treatment plan. As noted herein, the dental treatment plan may include orthodontic and/or restorative elements. The simulated outcome may correspond to estimated outcomes and/or intended outcomes of the dental treatment plan.
1. A computer-implemented method of simulating orthodontic treatment, the method comprising:
capturing a first 2D image, the first 2D image comprising a representation of a patient's face and a patient's teeth of a patient;
identifying one or more shapes associated with at least one of the patient's teeth;
building a parametric 3D model of the patient's teeth based on the 2D image, using one or more case-specific parameters for the one or more shapes associated with the at least one of the patient's teeth;
simulating an outcome of a dental treatment plan for the patient's teeth to produce a simulated outcome of the dental treatment plan;
modifying the parametric 3D model to provide a modified 3D model representing the simulated outcome of the dental treatment plan; and
rendering, using the modified 3D model, a second 2D image representing the patient's face, the second 2D image representing the patient's teeth in accordance with the simulated outcome of the dental treatment plan.
2. The computer-implemented method of claim 1 , wherein building the parametric 3D model includes:
finding edges of teeth and lips in the first 2D image;
aligning a parametric tooth model to the edges of the teeth and lips in the first 2D image to determine the case-specific parameters; and
storing the case-specific parameters of the parametric tooth model that align the parametric tooth model with the edges of the teeth, gingiva, and lips in the first 2D image.
3. The computer-implemented method of claim 1 , wherein rendering the second 2D image includes:
accessing the parametric 3D model of the patient's teeth;
projecting one or more teeth positions from the first 2D image onto the parametric 3D model;
mapping color data from the 2D image to corresponding locations on the parametric 3D model to generate textures for the parametric 3D model; and
using the textures as part of the second 2D image of the patient's face.
4. The computer-implemented method of claim 1 , simulating an outcome of the dental treatment plan provides a predetermined position of the patients' teeth, and wherein the predetermined position is based on an average position of a plurality of previous patients' teeth after dental treatment.
5. The computer-implemented method of claim 1 , simulating an outcome of the dental treatment plan provides a predetermined position of the patients' teeth, and wherein the predetermined position is based on an average position of a plurality of previous patients' teeth before dental treatment.
6. The computer-implemented method of claim 2 , further comprising:
aligning the parametric 3D tooth model to the edges of the teeth, gingiva, and lips in the first 2D image.
7. The computer-implemented method of claim 1 , wherein the first 2D image comprises a profile image representing a profile of the patient's face.
8. The computer-implemented method of claim 1 , wherein the simulated outcome of the dental treatment plan comprises an estimated outcome of the dental treatment plan.
9. The computer-implemented method of claim 1 , wherein the simulated outcome of the dental treatment plan comprises an intended outcome of the dental treatment plan.
10. The computer-implemented method of claim 1 , wherein the dental treatment plan comprises an orthodontic treatment plan, a restorative treatment plan, or some combination thereof.
11. The computer-implemented method of claim 1 , wherein capturing the first 2D image comprises: instructing a mobile phone or a camera to image the patient's face, or gathering the first 2D image from a storage device or a networked system.
12. The computer-implemented method of claim 1 , wherein building the parametric 3D model of the patient's teeth based on the 2D image, using one or more case-specific parameters for the one or more shapes associated with the at least one of the patient's teeth comprises:
coarsely aligning teeth represented in the 3D parametric model to the patient's teeth represented in the 2D image; and
executing an expectation step to determine a probability that a projection of a silhouette of the 3D parametric model matches one or more edges of the 2D image a first time.
13. The computer-implemented method of claim 12 , wherein building the parametric 3D model of the patient's teeth based on the 2D image, using one or more case-specific parameters for the one or more shapes associated with the at least one of the patient's teeth comprises:
executing a maximization step using a small angle approximation to linearize the rigid transformation of the teeth in the 3D model; and
executing the expectation step to determine a probability that a projection of a silhouette of the 3D parametric model matches the edges of the 2D image a second time.
14. The computer-implemented method of claim 13 , further comprising:
iterating though the expectation and maximization steps a first plurality of times to with a first subset of parameters; and
after iterating though the expectation and maximization steps the first plurality of times with the first subset of parameters of the 3D parametric model, iterating though the expectation and maximization steps a second plurality of times with the first and second subset of parameters.
15. A computer-implemented method of building a 3D model of teeth from a 2D image, the method comprising:
capturing a 2D image of a patient's face, including their teeth;
determining edges of teeth and gingiva within the first 2D image;
fitting the teeth in a 3D parametric model of teeth to the edges of the teeth and gingiva within the first 2D image, the 3D parametric model including case-specific parameters for the shape of the patient's teeth; and
determining the value of the case-specific parameters of the 3D parametric model based on the fitting.
16. The computer-implemented method of claim 15 , wherein:
fitting the teeth in the 3D parametric model of teeth to the edges of the teeth and gingiva within the first 2D image comprises:
coarsely aligning the teeth in the 3D parametric model to the teeth in the 2D image; and
executing an expectation step to determine a probability that a projection of a silhouette of the 3D parametric model matches the edges of the 2D image.
17. The computer-implemented method of claim 16 , wherein:
fitting the teeth in the 3D parametric model of teeth to the edges of the teeth and gingiva within the first 2D image further comprises:
executing a maximization step using a small angle approximation to linearize the rigid transformation of the teeth in the model; and
executing the expectation step to determine a probability that a projection of a silhouette of the 3D parametric model matches the edges of the 2D image again.
18. The computer-implemented method of claim 17 , further comprising:
iterating though the expectation and maximization steps a first plurality of times to with a first subset of parameters; and
after iterating though the expectation and maximization steps the first plurality of times with the first subset of parameters of the 3D parametric model, iterating though the expectation and maximization steps the second plurality of times with the first and second subset of parameters.
19. The computer-implemented method of claim 18 , wherein the first plurality of time is the same as the second plurality of times.
20. The computer-implemented method of claim 18 , wherein the first subset of case-specific parameters of the 3D parametric model are one or more of a scale factor and tooth location and orientation.
21. The computer-implemented method of claim 18 , wherein the second subset of parameters of the 3D parametric model are one or more of a tooth shape and tooth location and orientation.