Method and system for proposing and visualizing dental treatments
A method, system and computer readable storage media for proposing and visualizing orthodontic treatments. A patient may use a mobile device such as a smart phone to quickly and inexpensively visualize an orthodontic treatment for misaligned teeth and overlay it on a 3D model of the face and jaw, with a depth sensor being used to detect depth information. The orthodontic treatment may also be visualized in an augmented manner for a clinician. 3D data of the orthodontic treatment may be used for further analysis or fabrication of a treatment appliance.
1. A method of operating a system for proposal and augmented visualization of an orthodontic treatment, the system comprising: a display device for augmented visualization, wherein the display device comprises an augmented reality glass or HUD display; a mobile device which is configured for distance measurement; and at least one processor, the method comprising:
recording a three-dimensional image of a face with the mobile device to obtain a three-dimensional facial data record;
segmenting a mouth area of the three-dimensional facial data record;
obtaining a three-dimensional representation of actual teeth to be corrected;
establishing reference objects or reference frames in the three-dimensional representation of actual teeth to be corrected;
determining misalignments of the three-dimensional representation of actual teeth to be corrected by comparing the reference objects or reference frames with reference data;
determining an orthodontic treatment as a three-dimensional tooth data record based on the determined misalignments of the three-dimensional representation of actual teeth; and
overlaying the orthodontic treatment as an augmentation on the mouth area through the display device for augmented visualization such that the orthodontic treatment appears directly superimposed on the mouth area.
2. The method according to claim 1 , wherein the mobile device is further configured for motion tracking and/or for color measurement.
3. The method according to claim 1 , further comprising:
determining characteristic points within the three-dimensional facial data record;
determining a direction of orientation of the face in the three-dimensional facial data record based on the characteristic points; and
orienting said orthodontic treatment such that its direction matches the direction of orientation of the face in the three-dimensional facial data record.
4. The method according to claim 1 , further comprising designing a corrective appliance based on the orthodontic treatment.
5. The method according to claim 1 , wherein the three-dimensional representation of actual teeth is obtained through extraction from the three-dimensional image of the face or obtained from a scan.
6. The method according to claim 1 , further comprising:
detecting anatomical dental features in the three-dimensional representation of actual teeth to be corrected; and
determining misalignments of the three-dimensional representation of actual teeth to be corrected by comparing the anatomical dental features with reference data.
7. The method according to claim 6 , wherein the anatomical dental features include at least one of cusps, ridges, grooves and contact points.
8. The method according to claim 1 , further comprising continuously updating the three-dimensional facial data record based on data from the mobile device, including real time data tracking changes in facial expressions and/or patient movements.
9. The method according to claim 1 , wherein the orthodontic treatment is determined in real-time.
10. A non-transitory computer-readable storage medium storing a program which, when executed by a computer implemented system causes the system to perform the method steps according to claim 1 wherein the system includes:
a display device for augmented visualization, wherein the display device includes an augmented reality glass or HUD display;
a mobile device which is configured for distance measurement, and
at least one processor configured to perform the program that causes the at least one processor to:
record a three-dimensional image of a face with the mobile device to obtain a three-dimensional facial data record;
segment a mouth area of the three-dimensional facial data record;
obtain a three-dimensional representation of actual teeth to be corrected;
establish reference objects or reference frames in the three-dimensional representation of actual teeth to be corrected;
determine misalignments of the three-dimensional representation of actual teeth to be corrected by comparing the reference objects or reference frames with reference data;
determine an orthodontic treatment as a three-dimensional tooth data record based on the determined misalignments of the three-dimensional representation of actual teeth; and
overlay the orthodontic treatment as an augmentation on the mouth area through the display device for augmented visualization such that the orthodontic treatment appears directly superimposed on the mouth area.
11. The method according to claim 1 , wherein the reference data includes data obtained from ongoing patient treatments, past patient treatments, scientific literature and textbooks.
12. A system for visualizing an orthodontic treatment, the system comprising:
a display device for augmented visualization, wherein the display device comprises an augmented reality glass or HUD display;
a mobile device which is configured for distance measurement, and
at least one processor configured to perform the method comprising:
recording a three-dimensional image of a face with the mobile device to obtain a three-dimensional facial data record;
segmenting a mouth area of the three-dimensional facial data record;
obtaining a three-dimensional representation of actual teeth to be corrected;
establishing reference objects or reference frames in the three-dimensional representation of actual teeth to be corrected;
determining misalignments of the three-dimensional representation of actual teeth to be corrected by comparing the reference objects or reference frames with reference data;
determining an orthodontic treatment as a three-dimensional tooth data record based on the determined misalignments of the three-dimensional representation of actual teeth; and
overlaying the orthodontic treatment as an augmentation on the mouth area through the display device for augmented visualization such that the orthodontic treatment appears directly superimposed on the mouth area.
13. The system according to claim 12 , wherein the mobile device is further configured for motion tracking and/or for color measurement.