Video see-through augmented reality
In one embodiment, a method includes capturing, by a calibration camera, a calibration pattern displayed on a display of a video see-through AR system, where the calibration camera is located at an eye position for viewing content on the video see-through AR system. The method further includes determining, based on the captured calibration pattern, one or more system parameters of the video see-through AR system that represent combined distortion caused by a display lens of the video see-through AR system and caused by a camera lens of the see-through camera; determining, based on the one or more system parameters and on one or more camera parameters that represent distortion caused by the see-through camera, one or more display-lens parameters that represent distortion caused by the display lens; and storing the one or more system parameters and the one more display-lens parameters as a calibration for the system.
1 . A method comprising:
capturing, by a calibration camera that is not part of a video see-through AR system, a calibration pattern displayed on a display of a video see-through AR system comprising (1) a see-through camera that is distinct from the calibration camera (2) the display, configured to display content captured by the see-through camera and (3) a display lens configured to transmit light from the display, wherein:
the calibration camera is located at an eye position for viewing content on the video see-through AR system; and
the displayed calibration pattern corresponds to an image of a calibration pattern captured by a see-through camera of the video see-through AR system;
determining, based on the captured calibration pattern, one or more system parameters of the video see-through AR system that represent combined distortion of the video see-through AR system caused by (1) capturing an image for display by the see-through camera and (2) transmitting the displayed image through the display lens;
determining, based on an image of the calibration pattern captured by the see-through camera, one or more see-through camera parameters that represent distortion caused only by the see-through camera;
determining, based on the one or more system parameters and on the one or more see-through camera parameters that represent the distortion caused only by the see-through camera, one or more display-lens parameters that represent distortion caused only by transmitting the displayed image through the display lens; and
storing the one or more system parameters and the one more display-lens parameters as a calibration for the video see-through AR system.
2 . The method of claim 1 , wherein the video see-through AR system comprises a headset.
3 . The method of claim 1 , wherein:
the see-through camera of the video see-through AR system comprises one of a pair of stereoscopic cameras; and
the display of the video see-through AR system comprises one of a pair of displays.
4 . The method of claim 3 , further comprising performing the method of claim 1 for each display and see-through camera combination.
5 . The method of claim 3 , wherein the pair of stereoscopic cameras comprise one pair of a plurality of pairs of stereoscopic cameras of the video see-through AR system.
6 . The method of claim 1 , further comprising performing the method of claim 1 for each of a plurality of color channels.
7 . The method of claim 1 , wherein the combined distortion comprises one or more of: (1) geometric distortion or (2) chromatic aberration.
8 . The method of claim 1 , wherein the one or more system parameters of the video see-through AR system that represent combined distortion caused by the display lens of the video see-through AR system and caused by a camera lens of the see-through camera comprise a camera matrix for the video see-through camera and the display lens and a distortion model for the display lens and the video see-through camera.
9 . One or more non-transitory computer readable storage media embodying instructions and coupled to one or more processors that are operable to execute the instructions to:
capture, by a calibration camera that is not part of a video see-through AR system, a calibration pattern displayed on a display of a video see-through AR system comprising (1) a see-through camera that is distinct from the calibration camera (2) the display, configured to display content captured by the see-through camera and (3) a display lens configured to transmit light from the display, wherein:
the calibration camera is located at an eye position for viewing content on the video see-through AR system; and
the displayed calibration pattern corresponds to an image of a calibration pattern capture by a see-through camera of the video see-through AR system;
determine, based on the captured calibration pattern, one or more system parameters of the video see-through AR system that represent combined distortion of the video see-through AR system caused by (1) capturing an image for display by the see-through camera and (2) transmitting the displayed image through the display lens;
determine, based on an image of the calibration pattern captured by the see-through camera, one or more see-through camera parameters that represent distortion caused only by the see-through camera determine, based on the one or more system parameters and on the one or more see-through camera parameters that represent the distortion caused only by the see-through camera, one or more display-lens parameters that represent distortion caused only by transmitting the displayed image through the display lens; and
store the one or more system parameters and the one more display-lens parameters as a calibration for the video see-through AR system.
10 . The media of claim 9 , wherein:
the see-through camera of the video see-through AR system comprises one of a pair of stereoscopic cameras; and
the display of the video see-through AR system comprises one of a pair of displays.
11 . A system comprising:
one or more non-transitory computer readable storage media embodying instructions; and
one or more processors coupled to the non-transitory computer readable storage media, the one or more processors being operable to execute the instructions to:
capture, by a calibration camera that is not part of a video see-through AR system, a calibration pattern displayed on a display of a video see-through AR system comprising (1) a see-through camera that is distinct from the calibration camera (2) the display, configured to display content captured by the see-through camera and (3) a display lens configured to transmit light from the display, wherein:
the calibration camera is located at an eye position for viewing content on the video see-through AR system; and
the displayed calibration pattern corresponds to an image of a calibration pattern capture by a see-through camera of the video see-through AR system;
determine, based on the captured calibration pattern, one or more system parameters of the video see-through AR system that represent combined distortion of the video see-through AR system caused by (1) capturing an image for display by the see-through camera and (2) transmitting the displayed image through the display lens;
determine, based on an image of the calibration pattern captured by the see-through camera, one or more see-through camera parameters that represent distortion caused only by the see-through camera determine, based on the one or more system parameters and on the one or more see-through camera parameters that represent the distortion caused only by the see-through camera, one or more display-lens parameters that represent distortion caused only by transmitting the displayed image through the display lens; and
store the one or more system parameters and the one more display-lens parameters as a calibration for the video see-through AR system.
12 . The system of claim 11 , wherein:
the see-through camera of the video see-through AR system comprises one of a pair of stereoscopic cameras; and
the display of the video see-through AR system comprises one of a pair of displays.