Spatiotemporal calibration of RGB-D and displacement sensors

1. A method of calibrating an RGB-D device comprising:

providing an RGB-D camera;

providing a displacement sensor located proximate to the RGB-D camera;

providing a calibration target, the calibration target including a plurality of visual markers shown in a surface of the calibration target and a marker-free area, wherein the plurality of visual markers have known locations on the calibration target;

positioning the calibration target within a field of view of the RGB-D camera and the displacement sensor such that the RGB-D camera and displacement sensor capture data corresponding to the calibration target;

projecting a measuring point on the marker-free area of the calibration target with the displacement sensor, the measuring point located within a field of view of the RGB-D camera on the calibration target;

identifying a position of the measuring point projected by the displacement sensor on the calibration target with the RGB-D camera;

moving the calibration target a known distance relative to the RGB-D camera and displacement sensor and in relation to a location of the RGB-D camera and the displacement sensor;

receiving output data from the RGB-D camera and the displacement sensor, the output data including depth measurements of the calibration target from both the RGB-D camera and the displacement sensor;

aligning data from the RGB-D camera and data from the displacement sensor;

calibrating a relative configuration of the RGB-D camera and the displacement sensor based on the aligned data output from the RGB-D device and position information of the calibration target output from the displacement sensor.

2. The method of claim 1 , further comprising rotating the calibration target a known amount in relation to an orientation of the RGB-D device and further calibrating a relative configuration of the RGB-D device based on the known change of rotation of the calibration target.

3. The method of claim 1 , further comprising calibrating intrinsic camera parameters and lens distortion of the RGB-D device based on known locations of the visual markers on the calibration target.

4. The method of claim 1 , wherein aligning data from the RGB-D camera and data from the displacement sensor includes interpolating measurements of one of the RGB-D camera and displacement sensor.

5. The method of claim 1 , wherein the plurality of visual markers on the calibration target are arranged in an array or known positions.

6. The method of claim 1 , wherein the plurality of visual markers are shaped having a corner for ready identification of the visual markers by the RGB-D device.

7. A method of calibrating an RGB-D device having an RGB-D camera and a displacement sensor, the method comprising:

providing a calibration target, the calibration target including a plurality of visual markers shown in a surface of the calibration target and a marker-free area, wherein the plurality of visual markers has known locations on the calibration target;

positioning the calibration target within a field of view of the RGB-D device such that the RGB-D camera and displacement sensor capture data corresponding to the calibration target;

projecting a measuring area on the marker-free area of the calibration target with the displacement sensor, the measuring area located within a field of view of the RGB-D camera on the calibration target;

identifying a position of a measuring point within the measuring area projected by the displacement sensor on the calibration target with the RGB-D camera;

rotating the calibration target a known amount in relation to a location of the RGB-D device and the displacement sensor;

receiving output data from the RGB-D device including data from the RGB-D camera and the displacement sensor, the output data including depth measurements of the calibration target from both the RGB-D camera and the displacement sensor;

aligning data from the RGB-D camera and data from the displacement sensor;

calibrating intrinsic camera parameters and lens distortion of the RGB-D device based on known locations of the visual markers on the calibration target;

calibrating a relative configuration of the RGB-D camera and the displacement sensor based on aligned data output from the RGB-D device and known position information of the calibration target relative to the RGB-D camera and displacement sensor.

8. The method of claim 7 , wherein aligning data from the RGB-D camera and data from the displacement sensor includes interpolating measurements of one of the RGB-D camera and displacement sensor.

9. The method of claim 7 , wherein the plurality of visual markers on the calibration target are arranged in an array or known positions.

10. The method of claim 7 , wherein the plurality of visual markers are shaped having a corner for ready identification of the visual markers by the RGB-D device.

11. A method of calibrating an RGB-D device having an RGB-D camera and a displacement sensor, the method comprising:

providing a calibration target, the calibration target including a plurality of visual markers shown in a surface of the calibration target, wherein the plurality of visual markers has known locations on the calibration target;

positioning the calibration target within a field of view of the RGB-D device such that the RGB-D camera and displacement sensor capture data corresponding to the calibration target;

locating one or more corners of the calibration target with the RGB-D device;

projecting a measuring point on a non-marked area of the calibration target with the displacement sensor, the measuring point located within a field of view of the RGB-D camera;

locating the projected measuring point of the displacement sensor projected onto the calibration target with the RGB-D camera;

rotating the calibration target by a defined rotation amount and recording output data from the RGB-D device corresponding to rotation of the calibration target relative to the RGB-D device; and

solving an optimization problem of parameters of the RGB-D device to determine a relative relationship of the RGB-D device.

12. The method of claim 7 , further comprising moving and rotating the calibration target and receiving data from the RGB-D device corresponding to additional movement and rotation of the RGB-D device.

13. The method of claim 11 , wherein the plurality of visual markers on the calibration target are arranged in an array or known positions.

14. The method of claim 11 , wherein the plurality of visual markers are shaped having a corner for ready identification of the visual markers by the RGB-D device.