Image-based longitudinal analysis
Methods and apparatuses (including devices and systems, such as intraoral scanners and software) for analyzing images of a subject's teeth. In some examples these methods may include examining the same teeth or tooth regions across different times and/or different imaging modalities, including but not limited to visible light, infrared, florescent, etc. Also described herein are methods and apparatuses for using a single, master, control for automatically or semi-automatically simultaneously adjusting multiple imaging parameters across multiple images of a region taken with different imaging modalities and/or at different times.
1 . An intraoral scanning system, the system comprising:
a hand-held imaging sensor; and
a non-transitory, computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of:
identifying a region of interest (ROI) on a first three-dimensional (3D) model of a patient's teeth derived from a first set of intraoral scan data taken over a first time period;
generating or selecting a first representative image that includes the ROI from a plurality of images corresponding to the first set of intraoral scan data;
mapping the ROI from the first 3D model to a second 3D model derived from a second set of intraoral scan data taken over a second time period;
generating a second representative image corresponding to the second 3D model, wherein the second representative image corresponds to a same camera angle relative to the second 3D model as the first representative image relative to the first 3D model and includes a region of the second 3D model that corresponds to the ROI; and
displaying the first and second representative images.
2 . The system of claim 1 , wherein generating or selecting the first representative image comprises selecting the first representative image from the plurality of images, and further wherein the selection is based on a number of points defining the ROI and/or a camera angle relative to a surface containing the ROI.
3 . The system of claim 1 , wherein generating or selecting comprises selecting based on how close a camera angle relative to a surface containing the ROI is to an angle normal to the surface containing the ROI.
4 . The system of claim 1 , wherein identifying the ROI comprises using a trained neural network to select the ROI, wherein the trained neural network is trained to identify one or more clinical features from the first 3D model or the plurality of images corresponding to the first set of intraoral scan data.
5 . The system of claim 1 , wherein identifying the ROI comprises receiving a manually identified ROI from a user.
6 . The system of claim 1 , further comprising finding correspondence between the first 3D model and the second 3D model of the patient's teeth, before generating the second representative image.
7 . The system of claim 1 , wherein identifying the ROI on the first 3D model comprises identifying the ROI in the plurality of images corresponding to the first set of intraoral scan data by identifying points corresponding to one or more of a plurality of points defining the ROI in each of the plurality of images corresponding to the first set of intraoral scan data, further wherein generating or selecting the first representative image comprises selecting the first representative image from the first plurality of images corresponding to the first set of intraoral scan data.
8 . The system of claim 1 , wherein identifying the ROI on the first 3D model comprises identifying the ROI in the plurality of images corresponding to the first 3D model by identifying points corresponding to one or more of a plurality of points defining the ROI in each of the plurality of images corresponding to the first set of intraoral scan data, further wherein generating or selecting the first representative image comprises generating a synthetic image from the first 3D model and/or the plurality of images corresponding to the first set of intraoral scan data.
9 . The system of claim 8 , wherein an angle normal to a surface containing the ROI is factored into a camera angle of the synthetic image that is generated.
10 . The system of claim 1 , further comprising identifying, in a plurality of images corresponding to the second set of intraoral scan data, points corresponding to one or more of a plurality of points defining the ROI on the first 3D model.
11 . The system of claim 1 , wherein generating the second representative image corresponding to the second 3D model comprises generating a synthetic image using the same camera angle relative to the second 3D model as the first representative image.
12 . The system of claim 11 , wherein generating the synthetic image comprises using a trained neural network to generate the synthetic image.
13 . The system of claim 11 , wherein generating the synthetic image further comprises textural mapping the synthetic image.
14 . The system of claim 11 , wherein generating the synthetic image further comprises generating the synthetic image by one or more of: light field fusion (LLFF), Neural Radiance Fields (NERF), or scene representation networks (SRN).
15 . The system of claim 1 , wherein the instructions are further configured to cause the computer to carry out an additional step of modifying the first and/or second representative images so that the first and second representative images match more closely to each other.
16 . The system of claim 15 , wherein modifying the first and/or second representative images comprises adjusting one or more of: perspective, rotation, brightness, contrast, resolution, or color equalization.
17 . The system of claim 1 , wherein displaying the first and second representative images comprises displaying the first and second representative images side-by-side.
18 . The system of claim 1 , wherein displaying the first and second representative images comprises displaying the first and second representative images as part of a video loop.
19 . The system of claim 1 , further comprising comparing the first and second representative images and outputting data regarding differences between the representative images.
20 . The system of claim 19 , wherein comparing comprises comparing a caries detection severity map for the first representative image to a caries detection severity map for the second representative image.
21 . The system of claim 1 , wherein the first set of intraoral scan data of the patient's teeth comprises infrared images.
22 . An intraoral scanning system, the system comprising:
an imaging sensor; and
a non-transitory, computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of:
identifying a plurality of points defining a region of interest (ROI) on a first three-dimensional (3D) model of a patient's teeth derived from a first intraoral scan taken over a first time;
finding correspondence between the first 3D model and a second 3D model of the patient's teeth derived from a second intraoral scan taken over a second time;
finding points corresponding to the plurality of points defining the ROI in each of a plurality of images corresponding to the first 3D model, and generating a first representative image, wherein the first representative image includes a maximum of the plurality of points defining the ROI and/or has a corresponding camera angle that is approximately normal to a surface containing the ROI in the 3D model;
generating a second representative image corresponding to the second 3D model, wherein the second representative image has a same camera angle as the first representative image relative to the second 3D model as the first representative image and includes a region of the second 3D model that corresponds to the ROI;
modifying the first and/or second representative images to adjust one or more of: perspective, rotation or color equalization so that the first and second representative images match more closely to each other; and
displaying the first and second representative images.
23 . A non-transitory computer-readable medium comprising instructions which, when executed by one or more processors, causes the one or more processors to perform a method comprising:
identifying a region of interest (ROI) on a first three-dimensional (3D) model of a patient's teeth derived from a first set of intraoral scan data taken over a first time;
mapping the ROI in a plurality of images corresponding to the first 3D model of the first set of intraoral scan data;
generating or selecting a first representative image that includes the ROI from a plurality of images corresponding to the first set of intraoral scan data;
generating a second representative image corresponding to a second 3D model, wherein the second representative image corresponds to a same camera angle relative to the second 3D model as the first representative image and includes a region of the second 3D model that corresponds to the ROI;
modifying the first and/or second representative images so that the first and second representative images match more closely to each other; and
displaying the first and second representative images.