STEREOSCOPIC 3D CAMERA FOR VIRTUAL REALITY EXPERIENCE
Embodiments are disclosed for embedding calibration metadata for a stereoscopic video capturing device. The device captures a sequence of stereoscopic images by a plurality of image sensors and combines the captured sequence of stereoscopic images into a stereoscopic video sequence. The device further embeds calibration information into the stereoscopic video sequence in a real time as the sequence of stereoscopic images is being recorded. The calibration information can be used to correct distortion caused by hardware variances of individual video capturing devices. The corrected stereoscopic videos can be used to provide a virtual reality (VR) experience by immersing a user in a simulated environment.
1 . A method for embedding calibration metadata, comprising:
capturing a sequence of stereoscopic images by a plurality of image sensors of a stereoscopic video rerecording device;
combining the captured sequence of stereoscopic images into a stereoscopic video sequence; and
embedding calibration information into the stereoscopic video sequence in a real time as the sequence of stereoscopic images is being recorded.
2 . The method of claim 1 , wherein the step of embedding comprises:
storing the calibration information in a metadata header of a stereoscopic video file containing the stereoscopic video sequence in a real time as the sequence of stereoscopic images is being recorded and stored into the stereoscopic video file.
3 . The method of claim 1 , where the step of embedding comprises:
storing the calibration information in a subtitle channel or a closed caption channel of a stereoscopic video file containing the stereoscopic video sequence in a real time as the sequence of stereoscopic images is being recorded and stored into the stereoscopic video file.
4 . The method of claim 1 , where the step of embedding comprises:
embedding the calibration information into at least some of the stereoscopic images via an image steganography process in a real time as the sequence of stereoscopic images is being captured.
5 . The method of claim 1 , wherein the calibration information includes at least two sets of camera intrinsic calibration metadata, each set of camera intrinsic calibration metadata corresponds to a camera module of the stereoscopic video rerecording device including a wide-angle lens and an image processor.
6 . The method of claim 5 , wherein the set of camera intrinsic calibration metadata includes a plurality of focal lengths of a corresponding camera module, the focal lengths measures ability of the wide-angle lens of the corresponding camera module to bend light at different directions.
7 . The method of claim 5 , wherein the set of camera intrinsic calibration metadata includes coordinates of a distortion center of images captured by the wide-angle lens of the corresponding camera module.
8 . The method of claim 5 , wherein the set of camera intrinsic calibration metadata includes a plurality of distortion coefficients that describe levels of fisheye distortion as a function of a radius from a center of the captured image to an edge of the captured image.
9 . The method of claim 1 , wherein the calibration information includes a set of stereoscopic calibration metadata, which relates to relevant relationship between camera modules of the stereoscopic video rerecording device including wide-angle lenses and image processors.
10 . The method of claim 9 , wherein the set of stereoscopic calibration metadata is used to rectify the captured images for a left channel and a right channel such that rectified images of the left and right channels are in forms as though the rectified images were captured from the same image plane.
11 . The method of claim 9 , wherein the set of stereoscopic calibration metadata includes a rotation matrix that describes a rotational correction to align an image captured by one fisheye lens of the stereoscopic video rerecording device to another image captured by another fisheye lens of the stereoscopic video rerecording device.
12 . The method of claim 9 , wherein the set of stereoscopic calibration metadata includes a projection matrix vertically aligning an image captured by one fisheye lens of the stereoscopic video rerecording device to another image captured by another fisheye lens of the stereoscopic video rerecording device.
13 . The method of claim 1 , wherein the calibration information is embedded into the video sequence in a real time by a graphics processing unit (GPU).
14 . The method of claim 1 , wherein the calibration information includes inertial measurement data, location data, image sensor information, or lens distortion profile.