Stereoscopic 3D camera for virtual reality experience
Embodiments are disclosed for a stereoscopic device (also referred to simply as the “device”) that captures three-dimensional (3D) images and videos with a wide field of view and provides a virtual reality (VR) experience by immersing a user in a simulated environment using the captured 3D images or videos.
1. A method for mapping stereoscopic data from a three-dimensional space to a two-dimensional space and filtering motion data, comprising: obtaining a video stream from a stereoscopic camera that captures stereoscopic video streams, parameters representing a field of view of the stereoscopic camera, and parameters representing a setting of a lens or sensor of the stereoscopic camera; identifying a reference frame, a first set of frames before the reference frame, and a second set of frames after the reference frame from the video stream; comparing the first and second set of frames to characterize a first motion; filtering the first motion to generate a filtered motion, the filtered motion having a value corresponding to the reference frame; calculating a first matrix based on the filtered motion; calculating a second matrix representing a mapping from a three-dimensional space to a two-dimensional space associated with a plane of the stereoscopic camera in the absence of motion, wherein the second matrix is calculated using at least one of a focal length of the stereoscopic camera at a time the reference frame was captured and a principal point of the stereoscopic camera, wherein a respective item of intrinsic data in the three-dimensional space is estimated for each of the first and second set of frames; calculating a third matrix as an inverse of the first matrix; and applying the first matrix, second matrix, and third matrix together in a matrix operation to the reference frame to obtain a modified reference frame.
2. The method of claim 1 , wherein the method is applied on a plurality of reference frames, a corresponding set of frames before each of the plurality of reference frames, and a corresponding set of frames after each of the plurality of reference frames.
3. The method of claim 1 , wherein the method is applied on a plurality of stereoscopic cameras.
4. The method of claim 1 , wherein the method is applied on two stereoscopic cameras that independently capture left and right channels of a stereoscopic 30 video.
5. A method for video playback, comprising: obtaining a reference frame, a first set of frames before the reference frame, and a second set of frames after the reference frame from a camera wherein the camera is a stereoscopic camera that captures stereoscopic video streams; corn paring the first and second set of frames to characterize a first motion; filtering the first motion to generate a filtered motion, the filtered motion having a value corresponding to the reference frame; calculating a first matrix representing a mapping from a three-dimensional space to a two-dimensional space associated with a plane of the camera in the absence of motion, a second matrix as an inverse of the image's first matrix, and a third matrix relating to the filtered motion, wherein the first matrix is calculated using at least one of a focal length of the camera at a time the selected image was captured and a principal-point of the camera; and applying the first matrix, second matrix, and third matrix together in a matrix operation to the reference frame to obtain a modified reference frame.
6. The method of claim 5 , wherein the method is applied on a plurality of reference frames, a corresponding set of frames before each of the plurality of reference frames, and a corresponding set of frames after each of the plurality of reference frames.
7. The method of claim 5 , wherein the camera further captures parameters representing a field of view of the camera.
8. The method of claim 5 , wherein the camera can further capture parameters representing a setting of a lens or sensor assembly of the camera.
9. The method of claim 5 , wherein the camera includes an image sensor and lens.
10. The method of claim 5 , wherein two cameras independently capture left and right channels of the stereoscopic 3D video.
11. The method of claim 5 , wherein the plurality of cameras are configured with different exposure levels, white balances and/or color contrasts for high dynamic range (HDR) optimization.