Calibration system and method to align a 3D virtual scene and a 3D real world for a stereoscopic head-mounted display
A calibration platform may obtain measurements for aligning a real-world coordinate system and a display coordinate system. For example, the calibration platform may display, via an optical see-through head-mounted display (OST-HMD), a three-dimensional virtual object and receive, from a positional tracking device, information that relates to a current pose of a three-dimensional real-world object to be aligned with the three-dimensional virtual object. The calibration platform may record a three-dimensional position of a plurality of points on the three-dimensional real-world object based on the current pose of the three-dimensional real-world object, based on an indication that the plurality of points on the three-dimensional real-world object respectively corresponds with a plurality of points on the three-dimensional virtual object. Accordingly, based on the obtained measurements, the calibration platform may generate a transformation function to provide a mapping between three-dimensional points in the real-world coordinate system and three-dimensional points in the display coordinate system.
1. A device, comprising:
one or more memories; and
one or more processors, coupled to the one or more memories, configured to:
one or more instructions that, when executed by one or more processors, cause the one or more processors to:
generate a transformation function to provide a mapping between three-dimensional points in a real-world coordinate system to three-dimensional points in a display coordinate system corresponding to a three-dimensional display space of an optical see-through head-mounted display device based on a set of measurements gathered using a positional tracking device;
apply the transformation function to a first internal projection operator to produce a first corrected projection operator that the optical see-through head-mounted display device uses to display a first two-dimensional perspective image for a left eye;
apply the transformation function to a second internal projection operator to produce a second corrected projection operator that the optical see-through head-mounted display device uses to display a second two-dimensional perspective image for a right eye; and
display, via the optical see-through head-mounted display device, a three-dimensional image in the display coordinate system by presenting the first two-dimensional perspective image based on the first corrected projection operator and presenting the second two-dimensional perspective image based on the second corrected projection operator,
wherein the three-dimensional image in the display coordinate system is superimposed on an object in the real-world coordinate system.
2. The device of claim 1 , wherein the transformation function includes an affine transformation matrix to minimize a reprojection error associated with the set of measurements.
3. The device of claim 1 , wherein the transformation function includes a perspective transformation matrix to minimize a reprojection error associated with the set of measurements.
4. The device of claim 1 , wherein the transformation function includes an isometric transformation matrix to minimize a reprojection error associated with the set of measurements.
5. The device of claim 1 , wherein the positional tracking device is a camera coupled to the optical see-through head-mounted display device.
6. The device of claim 1 , wherein the positional tracking device is positioned at a particular location external to the optical see-through head-mounted display device.
7. The device of claim 1 , wherein the set of measurements is generated by:
discarding, from the set of measurements, one or more outliers based on a reprojection error of the set of measurements.
8. A non-transitory computer-readable medium storing instructions comprising:
one or more instructions that, when executed by one or more processors, cause the one or more processors to:
generate a transformation function to provide a mapping between three-dimensional points in a real-world coordinate system to three-dimensional points in a display coordinate system corresponding to a three-dimensional display space of an optical see-through head-mounted display device based on a set of measurements gathered using a positional tracking device;
apply the transformation function to a first internal projection operator to produce a first corrected projection operator that the optical see-through head-mounted display device uses to display a first two-dimensional perspective image for a left eye;
apply the transformation function to a second internal projection operator to produce a second corrected projection operator that the optical see-through head-mounted display device uses to display a second two-dimensional perspective image for a right eye; and
display, via the optical see-through head-mounted display device, a three-dimensional image in the display coordinate system by presenting the first two-dimensional perspective image based on the first corrected projection operator and presenting the second two-dimensional perspective image based on the second corrected projection operator,
wherein the three-dimensional image in the display coordinate system is superimposed on an object in the real-world coordinate system.
9. The non-transitory computer-readable medium of claim 8 , wherein the transformation function includes an affine transformation matrix to minimize a reprojection error associated with the set of measurements.
10. The non-transitory computer-readable medium of claim 8 , wherein the transformation function includes a perspective transformation matrix to minimize a reprojection error associated with the set of measurements.
11. The non-transitory computer-readable medium of claim 8 , wherein the transformation function includes an isometric transformation matrix to minimize a reprojection error associated with the set of measurements.
12. The non-transitory computer-readable medium of claim 8 , wherein the positional tracking device is a camera coupled to the optical see-through head-mounted display device.
13. The non-transitory computer-readable medium of claim 8 , wherein the positional tracking device is positioned at a particular location external to the optical see-through head-mounted display device.