3D video encoding and decoding methods and apparatus
Methods and apparatus relating to encoding and decoding stereoscopic (3D) image data, e.g., left and right eye images, are described. Various pre-encoding and post-decoding operations are described in conjunction with difference based encoding and decoding techniques. In some embodiments left and right eye image data is subject to scaling, transform operation(s) and cropping prior to encoding. In addition, in some embodiments decoded left and right eye image data is subject to scaling, transform operations(s) and filling operations prior to being output to a display device. Transform information and/or scaling information may be included in a bitstream communicating encoded left and right eye images. The amount of scaling can be the same for an entire scene and/or program.
1. A method, comprising:
determining, based on a frame pair comprising a first eye image and a second eye image, a first eye image shift and a second eye image shift,
wherein the first eye image comprises an image from a point of view of a first eye, the second eye image comprises an image from a point of view of a second eye, the first eye image, the first eye image shift corresponds to amount of transformation based on the first eye image and the second eye image;
generating a scaled and cropped first eye image based on the first eye image shift, wherein the scaled and cropped first eye image comprises a subset of the image data of the first eye image;
generating a scaled and cropped second eye image based on the second eye image shift, wherein the scaled and cropped second eye image comprises a subset of the image data of the second eye image;
encoding the scaled and cropped first eye image to obtain an encoded first eye image;
performing difference encoding on the scaled and cropped second eye image by referencing the scaled and cropped first eye image to obtain an encoded second eye image, wherein the encoded first eye image and the encoded second eye image comprise encoded image data; and
transmitting the encoded first eye image and data indicating at least one of the first eye shift, the second eye shift, and a scale at which one of the first eye image and second eye image was scaled.
2. The method of claim 1 , further comprising generating a bitstream for the encoded first eye image, encoded second eye image, and transformation information associated with scale and crop information for the encoded image data.
3. The method of claim 1 , further comprising storing the transformation information in association with the encoded image data.
4. The method of claim 1 , wherein generating a scaled and cropped first eye image comprises using a scaling parameter, wherein the scaling parameter is used for the frame pair and an additional frame pair.
5. The method of claim 1 , wherein generating a scaled and cropped second eye image comprises using a scaling parameter, wherein the scaling parameter is used for the frame pair and an additional frame pair.
6. The method of claim 1 , wherein the difference encoding comprises motion compensated prediction encoding.
7. The method of claim 1 , wherein the first eye image shift comprises a shift in a vertical direction and a horizontal direction.
8. A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
determine, based on a frame pair comprising a first eye image and a second eye image, a first eye image shift and a second eye image shift,
wherein the first eye image comprises an image from a point of view of a first eye, the second eye image comprises an image from a point of view of a second eye, the first eye image, the first eye image shift corresponds to amount of transformation based on the first eye image and the second eye image;
generate a scaled and cropped first eye image based on the first eye image shift, wherein the scaled and cropped first eye image comprises a subset of the image data of the first eye image;
generate a scaled and cropped second eye image based on the second eye image shift, wherein the scaled and cropped second eye image comprises a subset of the image data of the second eye image;
encode the scaled and cropped first eye image to obtain an encoded first eye image; and
perform difference encoding on the scaled and cropped second eye image by referencing the scaled and cropped first eye image to obtain an encoded second eye image, wherein the encoded first eye image and the encoded second eye image comprise encoded image data; and
transmit the encoded first eye image and data indicating at least one of the first eye shift, the second eye shift, and a scale at which one of the first eye image and second eye image was scaled.
9. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to generate a bitstream for the encoded first eye image, encoded second eye image, and transformation information associated with scale and crop information for the encoded image data.
10. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to store the transformation information in association with the encoded image data.
11. The non-transitory computer readable medium of claim 8 , wherein the computer readable code to generate a scaled and cropped first eye image further comprises computer readable code to use a scaling parameter, wherein the scaling parameter is used for the frame pair and an additional frame pair.
12. The non-transitory computer readable medium of claim 8 , wherein the computer readable code to generate a scaled and cropped second eye image further comprises computer readable code to use a scaling parameter, wherein the scaling parameter is used for the frame pair and an additional frame pair.
13. The non-transitory computer readable medium of claim 8 , wherein the difference encoding comprises motion compensated prediction encoding.
14. The non-transitory computer readable medium of claim 8 , wherein the first eye image shift comprises a shift in a vertical direction and a horizontal direction.
15. A system comprising:
one or more processors; and
one or more computer readable media comprising computer readable code executable by the one or more processors to:
determine, based on a frame pair comprising a first eye image and a second eye image, a first eye image shift and a second eye image shift,
wherein the first eye image comprises an image from a point of view of a first eye, the second eye image comprises an image from a point of view of a second eye, the first eye image, the first eye image shift corresponds to amount of transformation based on the first eye image and the second eye image;
generate a scaled and cropped first eye image based on the first eye image shift, wherein the scaled and cropped first eye image comprises a subset of the image data of the first eye image;
generate a scaled and cropped second eye image based on the second eye image shift, wherein the scaled and cropped second eye image comprises a subset of the image data of the second eye image;
encode the scaled and cropped first eye image to obtain an encoded first eye image; and
perform difference encoding on the scaled and cropped second eye image by referencing the scaled and cropped first eye image to obtain an encoded second eye image, wherein the encoded first eye image and the encoded second eye image comprise encoded image data; and
transmit the encoded first eye image and data indicating at least one of the first eye shift, the second eye shift, and a scale at which one of the first eye image and second eye image was scaled.
16. The system of claim 15 , further comprising computer readable code to generate a bitstream for the encoded first eye image, encoded second eye image, and transformation information associated with scale and crop information for the encoded image data.
17. The system of claim 15 , further comprising computer readable code to store the transformation information in association with the encoded image data.
18. The system of claim 15 , wherein the computer readable code to generate a scaled and cropped first eye image further comprises computer readable code to use a scaling parameter, wherein the scaling parameter is used for the frame pair and an additional frame pair.
19. The system of claim 15 , wherein the difference encoding comprises motion compensated prediction encoding.
20. The system of claim 15 , wherein the first eye image shift comprises a shift in a vertical direction and a horizontal direction.