Camera calibration of a telepresence system
A telepresence system may include a display configured to present three-dimensional images. The 3D images may be rendered from multiple images captured by multiple cameras that image an area from different viewpoints. Misalignment of any of the multiple cameras may negatively affect the rendering. Accordingly, the telepresence system may calibrate the cameras to compensate for any misalignment as part of the rending. This calibration may include capturing an image, or images, of a calibration target to determine the relative positions of the cameras. The disclosed telepresence system can perform this calibration online and using targets in fixed locations that are not easily noticeable to a user
1 . A telepresence system comprising:
a local display configured to present three-dimensional images to a local user in a local viewing-area;
a local set of cameras positioned relative to each other to capture images of the local viewing-area from multiple perspectives; and
a local processor communicatively coupled to the local display and the local set of cameras, the local processor configured by software instructions to perform a calibration process that includes:
determining a first set of relative positions of a first pair of cameras of the local set of cameras based on a target visible in a first set of images captured by the first pair of cameras;
determining a second set of relative positions of a second pair of cameras of the local set of cameras based on the target visible in a second set of images captured by the second pair of cameras;
determining a third set of relative positions of a third pair of cameras of the local set of cameras based on the first set of relative positions and the second set of relative positions; and
generating, or updating, a calibration for the local set of cameras to support rendering of stereoscopic three-dimensional images based on the first set of relative positions, the second set of relative positions, and the third set of relative positions, the calibration corresponding to misalignments of the local set of cameras from a set of target alignments.
2 . The telepresence system according to claim 1 , the target is on an accessory that is in a fixed location in the local viewing-area, the target facing the local set of cameras and facing away from the local user so that the accessory blocks the local user from viewing the target.
3 . The telepresence system according to claim 1 , wherein:
the telepresence system is configured to provide communication between the local user and a remote user; and
the calibration process occurs online without interrupting the communication between the local user and the remote user.
4 . The telepresence system according to claim 1 , wherein the calibration process is triggered by a temperature corresponding to the local set of cameras.
5 . The telepresence system according to claim 1 , wherein the target is on a speaker that is fastened to a desk surface of a desk in the local viewing-area, the target facing the local set of cameras and facing away from the local user so that the speaker blocks the local user from viewing the target.
6 . The telepresence system according to claim 1 , wherein the target is on a microphone that is fastened to a desk surface of a desk in the local viewing-area, the target facing the local set of cameras and facing away from the local user so that the microphone blocks the local user from viewing the target.
7 . The telepresence system according to claim 1 , wherein the target is on an equipment-mount fastened to a desk surface of a desk in the local viewing-area, the target facing the local set of cameras and facing away from the local user so that the equipment-mount blocks the local user from viewing the target.
8 . The telepresence system according to claim 1 , wherein the target is a mirror configured to reflect a pattern presented on the local display to the local set of cameras, the mirror facing away from the local user.
9 . The telepresence system according to claim 1 , wherein the target is not viewable by the third pair of cameras of the local set of cameras.
10 . The telepresence system according to claim 1 , wherein the target is located on a wall of the local viewing-area, the wall being behind the local user and facing the local set of cameras.
11 . The telepresence system according to claim 10 , wherein the target includes a pattern that obscures a purpose of the target from the local user.
12 . The telepresence system according to claim 1 , wherein the local set of cameras are mounted in a frame surrounding the local display.
13 . The telepresence system according to claim 12 , wherein the local set of cameras include:
the first pair of cameras of the local set of cameras that are positioned in a top portion of the frame above the local display;
the second pair of cameras of the local set of cameras that are positioned in a left portion the frame; and
the third pair of cameras of the local set of cameras that are positioned in a right portion the frame.
14 . A method for calibrating cameras of a telepresence system, the method comprising:
capturing images of a set of targets using a local set of cameras mounted in a frame of a local display, each target of the set of targets at fixed locations in a local viewing-area;
identifying a first set of images captured by a first pair of cameras of the local set of cameras that include a first target of the set of targets;
determine a first set of relative positions of the first pair of cameras based on the first target in the first set of images;
identifying a second set of images captured by a second pair of cameras of the local set of cameras that include a second target of the set of targets;
determine a second set of relative positions of the second pair of cameras based on the second target in the second set of images;
transitively computing a third set of relative positions of a third pair of cameras based on the first set of relative positions and the second set of relative positions; and
generating, or updating, a calibration to support rendering of stereoscopic three-dimensional images based on the first set of relative positions, the second set of relative positions, and the third set of relative positions, the calibration corresponding to misalignments of the local set of cameras from a set of target alignments.
15 . The method for calibrating the cameras of the telepresence system according to claim 14 , wherein:
the first target is located on a surface of an accessory that faces the local set of cameras and that faces away from a local user in the local viewing-area so that the accessory blocks the local user from viewing the first target; and
the second target is located on a wall of the local viewing-area, the wall behind the local user and facing the local set of cameras.
16 . The method for calibrating the cameras of the telepresence system according to claim 15 , wherein the first target or the second target includes a pattern that obscures a purpose of the first target or the second target from the local user.
17 . The method for calibrating the cameras of the telepresence system according to claim 14 , wherein the first target or the second target includes a mirror configured to reflect a pattern displayed on the local display to the local set of cameras.
18 . A telepresence system comprising:
a local viewing-area including:
a local set of cameras positioned relative to each other to capture images of the local viewing-area from multiple perspectives;
a target fixedly disposed on a surface of the local viewing-area; and
a local processor communicatively coupled to the local set of cameras, the local processor configured by local software instructions to perform a calibration process that includes:
capturing images of the target from at least two cameras of the local set of cameras;
determining based on the images of the target, relative positions between the at least two cameras; and
generating, or updating, a calibration file based on the relative positions; and
a remote viewing-area including:
a remote display configured to present three-dimensional images; and
a remote processor communicatively coupled to the local processor and the remote display, the remote processor configured by remote software instructions to perform a three-dimensional image rendering process that includes:
receiving images captured by the local set of cameras and the calibration file generated by the local processor;
generating adjusted images based on the calibration file, the adjusted images including corrected perspectives of the local viewing-area that compensate for misalignments of the local set of cameras; and
combining the adjusted images for display as the three-dimensional images on the remote display.
19 . The telepresence system according to claim 18 , wherein:
the target is hidden from view of a local user in the local viewing-area and/or includes a pattern that obscures a purpose the target from the local user.