Augmented reality enhancements for dental practitioners
A system comprises a scanner, an augmented reality (AR) display and a computing device. The scanner generates intraoral images of a dental arch and the AR display generates additional image data representative of a view from a wearer of the AR display. The computing device receives the intraoral images from the intraoral scanner, generates a virtual three-dimensional model of at least a portion of the dental arch from the intraoral images, receives the additional image data from the AR display, determines, from the additional image data, a region of the view that is outside of the dental arch, generates a visual overlay comprising the virtual three-dimensional model, and sends the visual overlay to the AR display. The AR display displays the visual overlay such that the virtual three-dimensional model is shown in the region of the view that is outside of the dental arch.
1. A system comprising:
a memory device; and
a processing device operatively coupled to the memory device, the processing device to:
receive an image of a dental arch, the image having been generated by an image capture device associated with an augmented reality (AR) display;
determine an intraoral procedure to be performed on a tooth of the dental arch;
detect a first area of interest in the image of the dental arch, wherein the first area of interest is associated with the intraoral procedure;
provide a visual overlay for output on the AR display identifying the first area of interest;
identify a change to the first area of interest during the intraoral procedure based on a comparison of first image data received from the image capture device to previous image data received from the image capture device; and
determine a first update to the visual overlay based on the change to the first area of interest.
2. The system of claim 1 , wherein the change to the first area of interest represents a removal of material from the tooth.
3. The system of claim 1 , wherein the intraoral procedure is a dental implant procedure, and the visual overlay comprises an indication of an insertion path for the dental implant procedure.
4. The system of claim 1 , wherein the intraoral procedure is a drilling procedure, and wherein:
to determine an update to the visual overlay the processing device is to determine that a depth of a drill present in the image has changed and generate an indication of a new depth of the drill; and
the processing device is further to determine that the new depth of the drill satisfies a depth threshold and transmit a notification to the drill to cause the drill to stop drilling.
5. The system of claim 1 , wherein the intraoral procedure is a drilling procedure that uses a drill, wherein the drill is present in the image of the dental arch, and wherein the processing device is further to:
determine from the image of the dental arch whether the drill is at a correct position and orientation; and
activate the drill responsive to determining that the drill is at the correct position and orientation.
6. The system of claim 1 , wherein the dental arch is one of an upper dental arch or a lower dental arch, wherein the visual overlay on the AR display comprises an occlusion map that shows contact points between the upper dental arch and the lower dental arch, wherein the intraoral procedure comprises grinding of a tooth, and wherein to determine the update to the visual overlay the processing device is further to:
identify a portion of the tooth that has been reduced from the grinding; and
determine a new occlusion map based on the portion of the tooth that has been reduced, the new occlusion map showing one or more new contact points between the upper dental arch and the lower dental arch.
7. The system of claim 6 , wherein to determine the new occlusion map the processing device is to:
update a three dimensional model of a patient's jaw, the three dimensional model comprising the upper dental arch and the lower dental arch; and
compute the new contact points between teeth of the upper dental arch and teeth of the lower dental arch based on the update to the three dimensional model.
8. The system of claim 1 , further comprising an infrared light source to provide infrared illumination of the dental arch.
9. The system of claim 1 , wherein the intraoral procedure is an intraoral scan of the dental arch, and wherein the processing device is further to:
receive image data generated by an intraoral scanner during an intraoral scan session; and
determine an area of the dental arch that has been scanned based on the image data, wherein the area of the dental arch that has been scanned comprises the first area of interest.
10. The system of claim 9 , wherein the first area of interest is depicted using a first visual indication, and wherein the processing device is further to:
determine an additional area of the dental arch that has not been scanned; and
add a second visual indication to the visual overlay corresponding to a position of the additional area.
11. The system of claim 9 , wherein the processing device is further to:
determine an additional area of the dental arch that has not been scanned; and
generate a three dimensional model of the dental arch using the intraoral scan to depict the area of the dental arch that has been scanned and using the image of the dental arch that was generated by the image capture device to depict the additional area of the dental arch that has not been scanned.
12. The system of claim 1 , wherein the processing device is further to:
perform a diagnosis of an area of the dental arch that has been scanned; and
add an indication of the diagnosis to the visual overlay.
13. The system of claim 1 , wherein the processing device is further to:
identify a second change to the first area of interest during the intraoral procedure based on a comparison of second image data received from the image capture device to the first image data received from the image capture device; and
determine a second update to the visual overlay based on the second change to the first area of interest.
14. A method comprising:
receiving an image of a dental arch, the image having been generated by an image capture device associated with an augmented reality (AR) display;
determining an intraoral procedure to be performed on a tooth of the dental arch;
detecting, by a processing device, a first area of interest in the image of the dental arch, wherein the first area of interest is associated with the intraoral procedure;
providing a visual overlay for output on the AR display identifying the first area of interest;
identifying, by the processing device, a change to the first area of interest during the intraoral procedure based on a comparison of first image data received from the image capture device to previous image data received from the image capture device; and
determining, by the processing device, a first update to the visual overlay based on the change to the first area of interest.
15. The method of claim 14 , wherein the change to the first area of interest represents a removal of material from the tooth.
16. The method of claim 14 , wherein the intraoral procedure is a dental implant procedure, and the visual overlay comprises an indication of an insertion path for the dental implant procedure.
17. The method of claim 14 , wherein the intraoral procedure is a drilling procedure, the method further comprising:
determining that a depth of a drill present in the image has changed;
generating an indication of a new depth of the drill, wherein the indication is represented in the visual overlay;
determining that the new depth of the drill satisfies a depth threshold; and
transmitting a notification to the drill to cause the drill to stop drilling.
18. The method of claim 14 , wherein the intraoral procedure is a drilling procedure that uses a drill, and wherein the drill is present in the image of the dental arch, the method further comprising:
determining from the image of the dental arch whether the drill is at a correct position and orientation; and
activating the drill responsive to determining that the drill is at the correct position and orientation.
19. The method of claim 14 , wherein the dental arch is one of an upper dental arch or a lower dental arch, wherein the visual overlay on the AR display comprises an occlusion map that shows contact points between the upper dental arch and the lower dental arch, and wherein the intraoral procedure comprises grinding of a tooth, the method further comprising:
identifying a portion of the tooth that has been reduced from the grinding; and
determining a new occlusion map based on the portion of the tooth that has been reduced, the new occlusion map showing one or more new contact points between the upper dental arch and the lower dental arch.
20. The method of claim 19 , wherein determining the new occlusion map comprises:
updating a three dimensional model of a patient's jaw, the three dimensional model comprising the upper dental arch and the lower dental arch; and
computing the new contact points between teeth of the upper dental arch and teeth of the lower dental arch based on the update to the three dimensional model.
21. The method of claim 14 , wherein the intraoral procedure is an intraoral scan of the dental arch, the method further comprising:
receiving image data generated by an intraoral scanner during an intraoral scan session; and
determining an area of the dental arch that has been scanned based on the image data, wherein the area of the dental arch that has been scanned comprises the first area of interest.
22. The method of claim 21 , wherein the first area of interest is depicted using a first visual indication, the method further comprising:
determining an additional area of the dental arch that has not been scanned; and
adding a second visual indication to the visual overlay corresponding to a position of the additional area.
23. The method of claim 21 , further comprising:
determining an additional area of the dental arch that has not been scanned; and
generating a three dimensional model of the dental arch using the intraoral scan to depict the area of the dental arch that has been scanned and using the image of the dental arch that was generated by the image capture device to depict the additional area of the dental arch that has not been scanned.
24. The method of claim 14 , further comprising:
performing a diagnosis of an area of the dental arch that has been scanned; and
adding an indication of the diagnosis to the visual overlay.
25. A non-transitory computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving an image of a dental arch, the image having been generated by an image capture device associated with an augmented reality (AR) display;
determining an intraoral procedure to be performed on a tooth of the dental arch;
detecting, by the processing device, a first area of interest in the image of the dental arch, wherein the first area of interest is associated with the intraoral procedure;
providing a visual overlay for output on the AR display identifying the first area of interest;
identifying, by the processing device, a change to the first area of interest during the intraoral procedure based on a comparison of first image data received from the image capture device to previous image data received from the image capture device; and
determining, by the processing device, a first update to the visual overlay based on the change to the first area of interest.