TRANSFORMING 3D VIDEO CONTENT TO MATCH VIEWER POSITION
Systems and methods for transforming 3D video content to match a viewer's position to provide a means to make constrained-viewpoint 3D video broadcasts more independent of viewer position. The 3D video display on a television is enhanced by taking 3D video that is coded assuming one particular viewer viewpoint, sensing the viewer's actual position with respect to the display screen, and transforming the video images as appropriate for the actual position. The process provided herein is preferably implemented using information embedded in an MPEG2 3D video stream or similar scheme to shortcut the computationally intense portions of identifying object depth that is necessary for the transformation to be performed.
1 . A process for transforming 3D video content to match viewer position, comprising the steps of
sensing the actual viewer's position, and
transforming a first sequence of right and left image pairs into a second sequence of right and left image pairs as function of viewer's sensed position, wherein the second right and left image pair produces a image that appears correct from a viewer's actual perspective.
2 . The process of claim 1 wherein the step of transforming comprises the steps of
receiving a sequence of right and left image pairs for each frame of video bitstream, the sequence of right and left image pairs being compressed by a method that reduces temporal and spatial redundancy, and
parsing from the sequence of right and left image pairs 2D dimensional images for right and left frames, and spatial information content and motion vectors.
3 . The process of claim 2 further comprising the steps of identifying points at which temporal redundancy become suspect within parsed spatial information.
4 . The process of claim 3 further comprising the steps of building a focus map as a function of DCT coefficient distribution within parsed spatial information, wherein the focus map groups areas of the image in which the degree of focus is similar.
5 . The process of claim 4 further comprising the step of validating motion vectors based on current values and stored values.
6 . The process of claim 5 further comprising the step of combining the motion vectors from the right and left frames to form a table of 3D motion vectors.
7 . The process of claim 6 further comprising the step of deriving a depth map for the current frame.
8 . The process of claim 7 wherein the step of deriving a depth map comprises the step of
generating three or more depth maps as a function of the points at which temporal redundancy becomes suspect, the focus map, the 3D motion vectors, the stored historic depth data and the 2D dimensional images for right and left frames,
comparing the three or more depth maps against discernible features from the 2D dimensional images for right and left frames,
selecting a depth map from the three or more depth maps, and
adding selected depth map to depth history.
9 . The process of claim 8 further comprising the steps of outputting the right and left frames as a function of the selected depth to provide a correct perspective to the viewer from viewer's actual position.
10 . The process of claim 9 wherein the step of outputting right and left frames comprising the steps of
transforming the selected depth map into 3D coordinate space, and
generating right and left frames from transformed depth map data wherein the right and left frames appear with appropriate perspective from the viewer's sensed position.
11 . The process of claim 10 further comprising the steps of
restoring missing portions of the image, and
displaying the image on a display screen.