Scanning system and calibration thereof
Disclosed is a non-calibrated scanning system to be modified to a calibrated scanning system based on generating a three-dimensional representation of a three-dimensional calibration-object.
1 . A non-calibrated scanning system to be modified to a calibrated scanning system based on generating a three-dimensional representation of a three-dimensional calibration-object, comprising:
a handheld intraoral scanning device, comprising:
a light source configured to emit light, wherein the scanning device is configured to transmit the light onto an object; and
an image-sensor comprising a plurality of sensor elements, wherein the image-sensor is configured to form a sequence of images of the object and the light as transmitted onto the object, wherein each image comprising a plurality of pixels; and
a processor configured for performing a calibration by performing the steps of:
importing a reference three-dimensional representation of the calibration-object in a real-world-coordinate-system;
registering the sequence of images into a registered sequence of images;
transforming, using a first transformation, the registered sequence of images and at least a set of the pixels from each of said images to a dataset of coordinates of the calibration-object in the scanning-device-coordinate-system;
transforming, using a second transformation, the dataset of coordinates of the calibration-object in the scanning-device-coordinate-system to a point-cloud of the calibration-object in a real-world-coordinate-system,
wherein the point-cloud of the calibration-object in the real-world coordinate-system is aligned to the reference three-dimensional representation of the calibration-object in the real-world-coordinate-system using an alignment-procedure, wherein the alignment procedure is based on minimizing a first difference between the reference three-dimensional representation and the point-cloud;
deriving, after using the alignment-procedure, a second difference between the point-cloud and the reference three-dimensional representation of the calibration-object in the real-world-coordinate-system;
modifying, based on the second difference, the second transformation to a calibrated transformation,
whereby the location of points in the point-clouds of the calibration-object in the real-world-coordinate-system is adjusted in three dimensions, and
whereby the non-calibrated scanning device is modified to a calibrated scanning device; and
a computer connected to the handheld intraoral scanning device, the computer comprising a display configured for:
displaying a graphical user-interface,
forming, in the graphical user-interface, a reference 3D-representation of the three-dimensional calibration-object,
displaying, in the graphical user interface, an indicator indicating where in the scan volume a user has collected data and where the user needs to scan further by a pointer or color on the reference 3D-representation, and
displaying, in the graphical user-interface, a notification indicating that the calibration of the handheld intraoral scanner device is completed.
2 . The non-calibrated scanning system according to claim 1 , wherein the alignment procedure is based on aligning the point-cloud to the reference three-dimensional representation by a point-to-plane registration or a point-to-point registration.
3 . The non-calibrated scanning system according to claim 1 , wherein:
the image-sensor is located in a plane normal to a z-direction in the scanning-device-coordinate-system, and
an optical element is configured to move relative to the image-sensor and along the z-direction, whereby the sequence of images of the object is formed while the optical element is moved.
4 . The non-calibrated scanning system according to claim 3 , wherein the processor is further configured to:
stack the registered sequence of images along the z-direction in the scanning-device-coordinate-system to form a volume,
wherein each image in the registered sequence of images comprises a layer in the volume, whereby the volume is comprised of a sequence of plurality of pixels;
compute a focus-measure for:
each of the plurality of pixels within the volume, or
each of a plurality of groups of pixels within the volume, wherein each of the plurality of group of pixels is formed by a subset of the plurality of pixels;
compute, based on the focus-measure, a maximum focus-measure, z m , for:
each of the plurality of pixels within the volume, whereby a subset of the plurality of pixels gets associated with z m , or
each of the plurality of group of pixels within the volume, whereby a subset of the plurality of pixels gets associated with z m ,
whereby, using the first transformation, the subset of the plurality of pixels that have been associated with z m is the at least set of pixels that are transformed to the dataset of coordinates of the object in the scanning-device-coordinate-system.
5 . The non-calibrated scanning system according to claim 3 , wherein the step of modifying the second transformation to a calibrated transformation, whereby the location of points in the point-cloud of the object in the real-world-coordinate-system is adjusted in three dimensions, is associated with a position along the z-direction of the optical element due to the sequence of images of the calibration-object is formed while the optical element is moved.
6 . The non-calibrated scanning system according to claim 5 , wherein the adjustment in three dimensions, as associated with a position along the z-direction of the optical element, is defined by a higher order polynomial for each of said positions along the z-direction.
7 . The non-calibrated scanning system according to claim 1 , wherein the step of modifying the second transformation to a calibrated transformation, whereby the location of points in the point-cloud of the object in the real-world-coordinate-system is adjusted in three dimensions, is performed by individually adjusting each of the points in the point-cloud.
8 . The non-calibrated scanning system according to claim 1 , wherein the step of modifying the second transformation to a calibrated transformation, whereby the location of points in the point-cloud of the object in the real-world-coordinate-system is adjusted in three dimensions, is performed by adjusting the three dimensional location simultaneously in three directions for each point in the point-cloud.
9 . The non-calibrated scanning system according to claim 1 , wherein the step of modifying the second transformation to a calibrated transformation, whereby the location of points in the point-cloud of the object in the real-world-coordinate-system is adjusted in three dimensions, is performed by adding a correction-vector, comprising three direction-components, to each of the points in the point-cloud.
10 . The non-calibrated scanning system according to claim 1 , wherein the step of modifying the second transformation to a calibrated transformation, is defined by a look-up-table.
11 . The non-calibrated scanning system according to claim 1 , wherein the step of importing a reference three-dimensional representation of the calibration-object in the real-world-coordinate-system is based on importing a high-precision three-dimensional representation of the calibration-object from a scanning system that differs from the non-calibrated scanning system.
12 . A system for obtaining a calibrated scanning system, comprising:
the non-calibrated scanning system according to claim 1 ; and
a three-dimensional calibration-object.
13 . The system according to claim 12 , wherein the three-dimensional calibration-object is in the shape of a dentition comprising a plurality of dental oral objects.
14 . A method of calibrating a non-calibrated scanning system by performing the steps of:
providing, the system according to claim 12 , thereby the non-calibrated scanning system and the three-dimensional calibration-object;
scanning the three-dimensional calibration-object using the non-calibrated scanning system, thereby scanning the three-dimensional calibration-object by using the scanning device, and thereby using the processor in the non-calibrated scanning system to perform the calibration, and
obtaining the calibrated scanning device according to claim 12 .
15 . A calibrated scanning system for generating a three-dimensional representation of an intraoral object, wherein the calibrated scanning system is provided by modifying the non-calibrated scanning system according to claim 1 , whereby the calibrated scanning system comprises:
a scanning device, comprising:
a light source configured to emit light, wherein the scanning device is configured to transmit the light onto an object; and
an image-sensor comprising a plurality of sensor elements, wherein the image-sensor is configured to form a sequence of images of the object and the light as transmitted onto the object, wherein each image comprising a plurality of pixels; and
a processor configured for performing the steps of:
registering the sequence of images;
transforming, using a first transformation, the registered sequence of images and at least a set of the pixels to a dataset of coordinates of the object in the scanning-device-coordinate-system;
transforming, using the calibrated transformation, the dataset of coordinates of the object in the scanning-device-coordinate-system to a point-cloud of the object in a real-world-coordinate-system, and
generating, based on the calibrated transformation, a calibrated three-dimensional representation of the object in the real-world-coordinate-system.
16 . A computer-implemented method for calibrating a handheld intraoral scanner device, displayed in a graphical user-interface on a screen of a computer connected to the handheld intraoral scanner device, the method, comprising the steps of:
obtaining reference-data of a reference 3D-representation of a three-dimensional calibration-object;
forming, in the graphical user-interface and based on the reference-data, a reference 3D-representation of the three-dimensional calibration-object;
obtaining, based on the handheld device being used by a user to scan the three-dimensional calibration-object, and from one or more device-to-real-world coordinate transformation(s) of 2D-images of the three-dimensional calibration-object, measurement-data;
aligning, using an alignment-procedure, the measurement-data to the reference-data to obtain alignment-data;
updating, based on the alignment-data, said one or more transformation(s), thereby calibrating the scanner device;
displaying, in the graphical user-interface, an indicator to the user from which the user is guided to continue scanning the three-dimensional calibration-object;
displaying, in the graphical user-interface, an indicator indicating where in the scan volume the user has collected data and where the user needs to scan further by a pointer or color on the reference 3D-representation; and
displaying, in the graphical user-interface, a notification indicating that the calibration of the handheld intraoral scanner device is completed.
17 . The computer implemented method according to claim 16 , wherein the method further comprising the step of forming, in the graphical user-interface and based on the alignment-data and/or said updated transformation(s), a 3D-representation of at least a part of the three dimensional calibration-object, such that said at least part can be seen in comparison to the displayed reference 3D-representation.
18 . The computer implemented method according to claim 16 , wherein the method comprising repeating one or more of said steps a plurality of times until a pre-defined criterion is satisfied.
19 . The computer implemented method according to claim 18 , wherein the step of displaying the notification is based on the pre-defined criteria.
20 . The computer implemented method according to claim 16 , wherein the step of aligning is initiated by a step of receiving, by the user in the user-interface, a point on the reference 3D-representation of the three-dimensional calibration-object, to assist the step of aligning.
21 . The computer implemented method according to claim 16 , wherein the indicator to the user from which the user is guided to continue scanning the three-dimensional calibration-object further indicates where on the three-dimensional calibration object, the user should continue scanning.
22 . The computer implemented method according to claim 16 , wherein the indicator is an absolute and/or relative measure of alignment-data, thereby indicating to the user of the hand-held device to which degree the user needs to continue scanning the three-dimensional calibration-object.