Photo realistic rendering of smile image after treatment

1. A method for digital treatment planning of a patient's teeth, the method comprising:

receiving a facial image of the patient that depicts the patient's teeth;

receiving a three-dimensional (3D) model of the patient's teeth;

coding the 3D model of the patient's teeth based on a plurality of attributes;

storing the coded 3D model;

identifying, by a neural network, reference points on the coded 3D model, wherein the reference points correspond to anatomical points;

generating, using the stored coded 3D model, a processed image of the patient's teeth, wherein anatomical attributes of the patient's teeth in the processed image are defined using the reference points; and

inserting the processed image of the patient's teeth into an oral cavity area of the facial image to form a composite image.

2. The method of claim 1 , wherein the anatomical points comprise one or more of a gingival apex of one or more teeth, a midpoint of an incisal edge of a tooth, an end of an incisal edge of a tooth, and a cusp tip of a tooth.

3. The method of claim 1 , wherein the anatomical points comprise a gingival apex of an anterior tooth.

4. The method of claim 1 , wherein the anatomical points comprise a cusp tip of a canine tooth.

5. The method of claim 1 , wherein the oral cavity area is defined at least in part by inner lip landmarks identified on the facial image of the patient.

6. The method of claim 1 , further comprising blurring the processed image of the patient's teeth using Gaussian blurring or alpha channel blurring, wherein the blurring is such that tooth structure in the processed image is not readily apparent to a human observer.

7. The method of claim 6 , wherein the blurring occurs after inserting the processed image of the patient's teeth into the oral cavity area of the facial image.

8. The method of claim 1 , further comprising simulating specular highlights on the processed image of the patient's teeth.

9. The method of claim 8 , wherein simulating specular highlights on the processed image of the patient's teeth comprises:

identifying bright locations on the patent's teeth in the facial image, wherein the bright locations are identified based on a comparison between a luminance value of a pixel and a threshold value;

determining a surface for a plurality of locations on the patient's teeth in the facial image;

identifying a set of normal angles at the bright location and luminance values for the set of normal angles; and

applying simulated specular highlights in the processed image based on the set of normal angles.

10. The method of claim 1 , further comprising providing the coded 3D model to the neural network.

11. The method of claim 2 , further comprising training the neural network to identify patient's teeth and contours of patient's teeth.

12. The method of claim 11 , wherein training the neural network to identify patient's teeth and contours of patient's teeth comprises providing a training dataset to the neural network, wherein the training dataset comprises a plurality of facial images.

13. The method of claim 11 , wherein training the neural network to identify patient's teeth comprises identifying at least one of a tooth type, a tooth location, or a tooth color.

14. The method of claim 1 , further comprising transmitting the composite image over a network connection to a mobile phone for display.

15. The method of claim 1 , further comprising displaying the composite image on a display device.

16. The method of claim 15 , further comprising displaying on the display device treatment options.

17. The method of claim 15 , further comprising displaying on the display device estimated results of an orthodontic treatment plan.

18. The method of claim 1 , further comprising determining a color palette of the 3D model of the patient's teeth.

19. The method of claim 1 , further comprising adjusting color values in the processed image of the patient's teeth.

20. The method of claim 1 , wherein the plurality of attributes comprises one or more of a brightness of the patients teeth at each pixel location, an angle of a surface of the 3D model with respect to a facial plane at each pixel location, and a dental structure type of the 3D model at each pixel location.

21. A system for digital treatment planning of a patient's teeth, the system comprising:

one or more processors; and

memory comprising instructions that, when executed by the one or more processors, cause the system to:

receive a facial image of the patient that depicts the patient's teeth;

receive a three-dimensional (3D) model of the patient's teeth;

code the 3D model of the patient's teeth based on a plurality of attributes;

store the coded 3D model;

identify, by a neural network, reference points on the coded 3D model, wherein the reference points correspond to anatomical points;

generate, using the stored coded 3D model, a processed image of the patient's teeth, wherein anatomical attributes of the patient's teeth in the processed image are defined using the reference points; and

insert the processed image of the patient's teeth into an oral cavity area of the facial image to form a composite image.

22. The system of claim 21 , wherein the anatomical points comprise one or more of a gingival apex of one or more teeth, a midpoint of an incisal edge of a tooth, an end of an incisal edge of a tooth, and a cusp tip of a tooth.

23. The system of claim 21 , wherein the anatomical points comprise a gingival apex of an anterior tooth.

24. The system of claim 21 , wherein the anatomical points comprise a cusp tip of a canine tooth.

25. The system of claim 21 , wherein the oral cavity area is defined at least in part by inner lip landmarks identified on the facial image of the patient.

26. The system of claim 21 , further comprising blurring the processed image of the patient's teeth using Gaussian blurring or alpha channel blurring, wherein the blurring is such that tooth structure in the processed image is not readily apparent to a human observer.

27. The system of claim 26 , wherein the blurring occurs after inserting the processed image of the patient's teeth into the oral cavity area of the facial image.

28. The system of claim 21 , further comprising simulating specular highlights on the processed image of the patient's teeth.

29. The system of claim 28 , wherein simulating specular highlights on the processed image of the patient's teeth comprises:

identifying bright locations on the patent's teeth in the facial image, wherein the bright locations are identified based on a comparison between a luminance value of a pixel and a threshold value;

determining a surface for a plurality of locations on the patient's teeth in the facial image;

identifying a set of normal angles at the bright location and luminance values for the set of normal angles; and

applying simulated specular highlights in the processed image based on the set of normal angles.

30. The system of claim 21 , further comprising providing the coded 3D model to the neural network.

31. The system of claim 21 , further comprising training the neural network to identify patient's teeth and contours of patient's teeth.

32. The system of claim 31 , wherein training the neural network to identify patient's teeth and contours of patient's teeth comprises providing a training dataset to the neural network, wherein the training dataset comprises a plurality of facial images.

33. The system of claim 31 , wherein training the neural network to identify patient's teeth comprises identifying at least one of a tooth type, tooth location, or a tooth color.

34. The system of claim 21 , further comprising transmitting the composite image over a network connection to a mobile phone for display.

35. The system of claim 21 , further comprising displaying the composite image on a display device.

36. The system of claim 35 , further comprising displaying on the display device treatment options.

37. The system of claim 35 , further comprising displaying on the display device estimated results of an orthodontic treatment plan.

38. The system of claim 21 , further comprising determining a color palette of the 3D model of the patient's teeth.

39. The system of claim 21 , further comprising adjusting color values in the processed image of the patient's teeth.

40. The system of claim 21 , wherein the plurality of attributes comprises one or more of a brightness of the patient's teeth at each pixel location, an angle of a surface of the 3D model with respect to a facial plane at each pixel location, and a dental structure type of the 3D model at each pixel location.