Deep learning for automated smile design
A method for displaying teeth after planned orthodontic treatment in order to show persons how their smiles will look after the treatment. The method includes receiving a digital 3D model of teeth or rendered images of teeth, and an image of a person such as a digital photo. The method uses a generator network to produce a generated image of the person showing teeth of the person, the person's smile, after the planned orthodontic treatment. The method uses a discriminator network processing input images, generated images, and real images to train the generator network through deep learning models to product a photo-realistic image of the person after the planned treatment.
1 . A computer-implemented method for displaying teeth after planned orthodontic treatment, comprising:
receiving, by a computing device, a digital three-dimension (3D) model of teeth or rendered images of teeth, and an image of a person;
processing, by a generator network implemented on the computing device, the digital 3D model and the image of the person to produce a photo-realistic generated image of the person showing teeth of the person after the planned orthodontic treatment of the teeth;
training the generator network using adversarial learning by a discriminator network that processes input images, generated images, and real images to improve image realism; and
displaying, on a graphical user interface of the computer device, the generated image of the person showing the results of the planned orthodontic treatment.
2 . The method of claim 1 , further comprising receiving a final alignment or stage of the teeth after the planned orthodontic treatment.
3 . The method of claim 2 , further comprising blocking out teeth of the person in the received image.
4 . The method of claim 3 , wherein using the generator network comprises filling in the blocked out teeth in the received image with the final alignment or stage of the teeth.
5 . The method of claim 1 , further comprising using a feature extracting network to extract features from the digital 3D model of teeth or rendered images of teeth for the generator network.
6 . The method of claim 1 , further comprising using a feature extracting network to extract features from the digital 3D model of teeth or rendered images of teeth for the discriminator network.
7 . The method of claim 1 , wherein the image comprises a digital photo.
8 . The method of claim 1 , wherein when the discriminator network is provided with real images, the discriminator network classifies the input images as real in order to train the generator network.
9 . The method of claim 1 , wherein when the discriminator network is provided with generated images, the discriminator network classifies the input images as fake in order to train the generator network.
10 . The method of claim 1 , wherein the planned orthodontic treatment comprises a final stage or setup.
11 . The method of claim 1 , wherein the planned orthodontic treatment comprises an intermediate stage or setup.
12 . A system for displaying teeth after planned orthodontic treatment, comprising a processor configured to:
receive, by the system, a digital three-dimension (3D) model of teeth or rendered images of teeth, and an image of a person;
process, by a generator network implemented on the computing device, the digital 3D model and the image of the person to produce a photo-realistic generated image of the person showing teeth of the person after the planned orthodontic treatment of the teeth;
train the generator network using adversarial learning by a discriminator network that processes input images, generated images, and real images to improve image realism; and
display, on a graphical user interface of the system, the generated image of the person showing the results of the planned orthodontic treatment.
13 . The system of claim 12 , wherein the processor is further configured to receive a final alignment or stage of the teeth after the planned orthodontic treatment.
14 . The system of claim 13 , wherein the processor is further configured to block out teeth of the person in the received image.
15 . The system of claim 14 , wherein to use the generator network, the processor is configured to fill in the blocked out teeth in the received image with the final alignment or stage of the teeth.
16 . The system of claim 12 , wherein the processor is further configured to use a feature extracting network to extract features from the digital 3D model of teeth or rendered images of teeth for the generator network.
17 . The system of claim 12 , wherein the processor is further configured to use a feature extracting network to extract features from the digital 3D model of teeth or rendered images of teeth for the discriminator network.
18 . The system of claim 12 , wherein the image comprises a digital photo.
19 . The system of claim 12 , wherein when the discriminator network is provided with real images, the discriminator network classifies the input images as real in order to train the generator network.
20 . The system of claim 12 , wherein when the discriminator network is provided with generated images, the discriminator network classifies the input images as fake in order to train the generator network.