Method for generating a virtual 4D head and teeth
It refers to the digitalization in three dimensions of a patient's head, intraoral maxillary and mandibular regions and bones. It includes integration of the facial movement to simulate the aesthetical and functional result before medical treatment, which could include surgery, and orthodontic and/or prosthodontic treatments. It comprises: creation of a 3D virtual model of the patient's head ( 2 ), creation of a 3D model of the patient's denture ( 4 ), fusion and anchoring of the 3D denture model ( 4 ) into the 3D head model ( 2 ), animation of the 3D head model ( 2 ) from 2D images ( 5 ) of the patient, resulting in an animated 3D model ( 6 ), animation of the 3D denture model ( 4 ) from 2D images ( 5 ) of the patient, and fusion with the 3D animated model ( 6 ), obtaining an animated 3D model of the patient's head and denture ( 7 ).
1. A method for the generation of a head and virtual denture of a patient, wherein said method comprises:
data preparation, which in turn comprises:
generation of a 3D virtual model of the head, obtained from images of a patient under study,
generation of a 3D virtual denture model of a patient's denture, obtained from data of the patient's teeth morphology,
determining the position of the 3D virtual denture model with respect to the 3D virtual model of the head,
fusion and anchoring of the 3D virtual denture model inside the 3D virtual model of the head in a neutral state,
training an artificial neural network on a database of 2D images of people performing different facial gestures and its corresponding 3D model with denture included in its corresponding position, learning the artificial neural network to estimate a movement of the 3D virtual denture model with respect to the 3D virtual model of the head,
integration of the movement of the 3D virtual model of the head and 3D virtual denture model, which in turn comprises:
animation of the 3D virtual model of the head from a set of 2D images of the patient, resulting in an animated 3D model of the patient's head,
independently positioning the 3D virtual denture model inside the animated 3D model, by applying an amount of movement to the 3D virtual denture model estimated from the set of 2D images of the patient by using the artificial neural network trained during the data preparation, and
fusion of the 3D virtual denture model with the animated 3D model, resulting in a fully animated 3D model consisting of the patient's head and denture.
2. The method according to claim 1 , wherein the images of the patient under study are images selected among photographs, video and/or a 3D scan of the patient's head.
3. The method according to claim 1 , wherein the 3D virtual denture model comprising a model of the upper denture, a model of the lower denture and information of their relative position and occlusion.
4. The method according to claim 1 , wherein the data of the patient's teeth morphology is obtained from an intraoral scanner and/or digitized conventional impressions.
5. The method according to claim 1 , wherein the data preparation phase additionally comprises a step during which methods of digital waxing, orthodontic movements and/or surgeries planification are applied to the 3D virtual denture model.
6. The method according to claim 1 , wherein the set of 2D images of the patient are taken from a previously recorded video of the patient.
7. The method according to claim 1 , wherein the set of 2D images of the patient are taken in real-time using a camera.
8. The method according to claim 1 , wherein the animation of the 3D virtual model of the head from 2D images of the patient further comprises:
estimating, with facial recognition techniques, information on the position and movement of the patient in the set of 2D images of the patient,
transferring said information to a generic 3D head model, and
transferring said movement from the generic 3D head model to the 3D virtual model of the head.
9. The method according to claim 1 , wherein in the positioning of the 3D denture model from the set of 2D images of the patient, a neutral position of the 3D virtual denture model is taken into account to compute how much movement needs to be applied to the 3D virtual denture model as a function of how much movement is observed in the set of 2D images of the patient.
10. The method according to claim 1 , further comprising a step of in-painting and texture addition to the fully animated 3D model.
11. The method according to claim 1 , further comprising a final step of projection of the fully animated 3D model to 2D, obtaining a simulation video.
12. A data-processing system for the generation of a head and virtual denture of a patient, said data-processing system comprising:
a processor configured to:
generate a 3D virtual model of the head, obtained from images of a patient under study,
generate a 3D virtual denture model of the patient's denture, obtained from data of the patient's teeth morphology,
determine the position of the 3D virtual denture model with respect to the 3D virtual model of the head,
fusion and anchor the 3D virtual denture model inside the 3D virtual model of the head in a neutral state,
train an artificial neural network on a database of 2D images of people performing different facial gestures and its corresponding 3D model with denture included in its corresponding position, learn the artificial neural network to estimate a movement of the 3D virtual denture model with respect to the 3D virtual model of the head,
animate the 3D virtual model of the head from 2D images of the patient, resulting in an animated 3D model of the patient's head,
independently position the 3D virtual denture model inside the animated 3D model, by applying an amount of movement to the 3D virtual denture model estimated from 2D images of the patient by using the artificial neural network, and
merge the 3D virtual denture model with the animated 3D model, resulting in a fully animated 3D model consisting of the patient's head and denture.
13. A non-transitory computer-readable storage medium storing the steps for the generation of a head and virtual denture of a patient, said non-transitory computer-readable storage medium configured to be executed by a processor to carry out the steps of the method of claim 1 .