Photograph-based assessment of dental treatments and procedures
The current document is directed to methods and systems for monitoring a dental patient's progress during a course of treatment. A three-dimensional model of the expected positions of the patient's teeth can be projected, in time, from a three-dimensional model of the patient's teeth prepared prior to beginning the treatment. A digital camera is used to take one or more two-dimensional photographs of the patient's teeth, which are input to a monitoring system. The monitoring system determines virtual-camera parameters for each two-dimensional input image with respect to the time-projected three-dimensional model, uses the determined virtual-camera parameters to generate two-dimensional images from the three-dimensional model, and then compares each input photograph to the corresponding generated two-dimensional image in order to determine how closely the three-dimensional arrangement of the patient's teeth corresponds to the time-projected three-dimensional arrangement.
1. A computer-implemented method of assessing a dental treatment for a patient's teeth, comprising:
receiving a two-dimensional image of the patient's teeth taken at a particular time during a course of the dental treatment;
receiving a three-dimensional model representing an expected configuration of the patient's teeth at the particular time;
projecting, in time, expected positions of the patient's teeth using the three-dimensional model to obtain a time-projected three-dimensional model of expected positions of the patient's teeth;
generating an expected two-dimensional image from the time-projected three-dimensional model using virtual-camera parameters to model a configuration of a camera used to capture the received two-dimensional image of the patient's teeth;
comparing the received two-dimensional image with the expected two-dimensional image generated from the time-projected three-dimensional model by calculating a fitness metric, wherein calculating the fitness metric comprises calculating a cost for each overlapping pixel along overlaid teeth contours of the received and expected two-dimensional images, wherein calculating the cost for each of one or more of the overlapping pixels comprises calculating a dot product of gradients for the overlapping pixel divided by the product of magnitudes of the gradients;
determining whether a configuration of the patient's teeth is within a threshold level of correspondence to the expected configuration of the patient's teeth based on the fitness metric; and
responsive to the determination as to whether the configuration of the patient's teeth is within the threshold level, providing an indication relating to the dental treatment.
2. The method of claim 1 , wherein comparing the received two-dimensional image with the expected two-dimensional image comprises
overlaying the teeth contours of the received and expected two-dimensional images.
3. The method of claim 1 , wherein calculating the fitness metric further comprises calculating a sum of the costs of the overlapping pixels along overlaid teeth contours divided by a length of the overlaid teeth contours in pixels.
4. The method of claim 1 , further comprising applying a Gaussian filter to the received two-dimensional image of the patient's teeth.
5. The method of claim 1 , wherein calculating the cost for each overlapping pixel comprises constructing a compact region along the overlaid teeth contours.
6. The method of claim 5 , wherein the compact region has a square or a rectangle shape.
7. The method of claim 1 , wherein providing the indication relating to the dental treatment comprises providing an indication that the dental treatment is on track or off track.
8. The method of claim 1 , wherein generating the expected two-dimensional image comprises matching an orientation and position of an origin a camera coordinate system used to take the two-dimensional image of the patient's teeth with a center of an image plane of a virtual camera used to generate the expected two-dimensional image.
9. The method of claim 1 , further comprising:
determining an iteration variable N based on metadata associated with the received two-dimensional image; and
iteratively modifying the virtual-camera parameters N times to produce the expected two-dimensional image from the time-projected three-dimensional model.
10. The method of claim 9 , wherein the metadata include one or more of: a text label, an indication of a standard type of view represented by the two-dimensional image, an image size, a date and time information, an indication of a camera model and make, an indication of a camera orientation, an indication of an aperture, an indication of a shutter speed, an indication of a focal length, an indication of a metering mode, and an indication of an image speed.
11. A dental-treatment monitoring system comprising:
one or more processors;
memory coupled to the one or more processors, wherein the memory includes computer-program instructions that, when executed by the one or more processors, cause the system to perform a computer-implemented method comprising:
receiving a two-dimensional image of a patient's teeth taken at a particular time during a course of a dental treatment;
receiving a three-dimensional model representing an expected configuration of the patient's teeth at the particular time;
projecting, in time, expected positions of the patient's teeth using the three-dimensional model to obtain a time-projected three-dimensional model of expected positions of the patient's teeth;
generating an expected two-dimensional image from the time-projected three-dimensional model using virtual-camera parameters to model a configuration of a camera used to capture the received two-dimensional image of the patient's teeth;
comparing the received two-dimensional image with the expected two-dimensional image generated from the time-projected three-dimensional model by calculating a fitness metric, wherein calculating the fitness metric comprises calculating a cost for each overlapping pixel along overlaid teeth contours of the received and expected two-dimensional images, wherein calculating the cost for each of one or more of the overlapping pixels comprises calculating a dot product of gradients for the overlapping pixel divided by the product of magnitudes of the gradients;
determining whether a configuration of the patient's teeth is within a threshold level of correspondence to the expected configuration of the patient's teeth based on the fitness metric; and
responsive to the determination as to whether the configuration of the patient's teeth is within the threshold level, providing an indication relating to the dental treatment.
12. The system of claim 11 , wherein comparing the received two-dimensional image with the expected two-dimensional image comprises
overlaying the teeth contours of the received and expected two-dimensional images.
13. The system of claim 11 , wherein calculating the fitness metric further comprises calculating a sum of the costs of the overlapping pixels along overlaid teeth contours divided by a length of the overlaid teeth contours in pixels.
14. The system of claim 11 , wherein calculating the cost of each of the one or more of the overlapping pixels comprises constructing a compact region along the overlaid teeth contours.
15. The system of claim 14 , wherein the compact region has a square or a rectangle shape.
16. A computer-implemented method of assessing a dental treatment for a patient's teeth, comprising:
receiving a two-dimensional image of the patient's teeth taken at a particular time during a course of the dental treatment;
receiving a three-dimensional model representing an expected configuration of the patient's teeth at the particular time;
projecting, in time, expected positions of the patient's teeth using the three-dimensional model to obtain a time-projected three-dimensional model of expected positions of the patient's teeth;
generating an expected two-dimensional image from the time-projected three-dimensional model using virtual-camera parameters to model a configuration of a camera used to capture the received two-dimensional image of the patient's teeth;
comparing the received two-dimensional image with the expected two-dimensional image generated from the time-projected three-dimensional model by calculating a fitness metric, wherein calculating the fitness metric comprises calculating a cost for each overlapping pixel along overlaid teeth contours of the received and expected two-dimensional images, wherein calculating the cost for each of one or more of the overlapping pixels comprises calculating a dot product of gradients for the overlapping pixel divided by the product of magnitudes of the gradients;
determining whether a configuration of the patient's teeth is within a threshold level of correspondence to the expected configuration of the patient's teeth from the fitness metric; and
responsive to the determination as to whether the configuration of the patient's teeth is within the threshold level, providing an indication relating to the dental treatment.
17. The method of claim 16 , further comprising:
determining an iteration variable N based on metadata associated with the received two-dimensional image; and
iteratively modifying the virtual-camera parameters N times to produce the expected two-dimensional image from the time-projected three-dimensional model.
18. The method of claim 17 , wherein the metadata include one or more of: a text label, an indication of a standard type of view represented by the two-dimensional image, an image size, a date and time information, an indication of a camera model and make, an indication of a camera orientation, an indication of an aperture, an indication of a shutter speed, an indication of a focal length, an indication of a metering mode, and an indication of an image speed.
19. The method of claim 16 , wherein calculating the fitness metric further comprises calculating a sum of the costs of the overlapping pixels along overlaid teeth contours divided by a length of the overlaid teeth contours in pixels.
20. The system of claim 11 , wherein providing the indication relating to the dental treatment comprises providing an indication that the dental treatment is on track or off track.