Encoding and decoding architectures for format compatible 3D video delivery
Encoding and decoding architectures for 3D video delivery are described, such as 2D compatible 3D video delivery and frame compatible 3D video delivery. The architectures include pre-processing stages to pre-process the output of a base layer video encoder and/or decoder and input the pre-processed output into an enhancement layer video encoder and/or decoder of one or more enhancement layers. Multiplexing methods of how to combine the base and enhancement layer videos are also described.
1. A two-dimensional (2D) compatible or frame compatible three-dimensional (3D) video decoding method, comprising:
performing a first enhancement layer video decoding of an initial plurality of encoded images or frames, each image or frame comprising two or more views, to form first enhancement layer reconstructed images or frames;
storing, in a first enhancement layer decoding reference buffer, the first enhancement layer reconstructed images or frames;
pre-processing first enhancement layer video reconstructed images or frames;
storing, in second enhancement layer decoding reference buffer, the pre-processed first enhancement layer reconstructed images or frames for second enhancement layer video decoding;
performing a second enhancement layer video decoding the initial plurality of images or frames based on the pre-processed first enhancement layer reconstructed images or frames; and
decomposing a checkerboard interleaved frame into upconverted component planes, wherein all of the two or more views are decoded and processed in each of the first enhancement layer and the second enhancement layer.
2. The two-dimensional (2D) compatible or frame compatible three-dimensional (3D) video decoding method of claim 1 , wherein at least one encoder pre-processing module iii) receives input from a first enhancement layer decoding reference buffer and transmits one output to an input of one second enhancement layer decoding reference buffer.
3. The method of claim 1 , wherein the two or more views are interleaved in a checkerboard pattern.
4. A decoder for decoding a two-dimensional (2D) compatible or frame compatible three-dimensional (3D) video signal, the decoder comprising:
a first decoding component function for performing a first enhancement layer video decoding an initial plurality of encoded images or frames, each image or frame comprising two or more views, to form a first enhancement layer reconstructed images or frames;
a first storage component function for storing, in a first enhancement layer decoding reference buffer, the first enhancement layer reconstructed images or frames;
a pre-processing component function for pre-processing first enhancement layer video reconstructed images or frames;
a second storage component function for storing, in a second enhancement layer decoding reference buffer, the pre-processed first enhancement layer reconstructed images or frames for second enhancement layer video decoding; and
a second decoding component function for performing second enhancement layer video decoding the initial plurality of images or frames based on the pre-processed first enhancement layer reconstructed images or frames, and for decomposing a checkerboard interleaved frame into upconverted component planes, wherein all of the two or more views are decoded and processed in each of the first enhancement layer and the second enhancement layer.
5. The decoder of claim 4 , wherein the two or more views are interleaved in a checkerboard pattern.
6. An apparatus for decoding a two-dimensional (2D) compatible or frame compatible three-dimensional (3D) video signal, the apparatus comprising:
a first decoding component for performing first enhancement layer video decoding an initial plurality of encoded images or frames, each image or frame comprising two or more views, to form first enhancement layer reconstructed images or frames;
a first storage component for storing, in a first enhancement layer decoding reference buffer, the first enhancement layer reconstructed images or frames;
a pre-processing component for pre-processing first enhancement layer video reconstructed images or frames refining views of the first enhancement layer video reconstructed images or frames for prediction;
a second storage component for storing, in a second enhancement layer decoding reference buffer, the pre-processed first enhancement layer reconstructed images or frames for second enhancement layer video decoding; and
a second decoding component for performing second enhancement layer video decoding the initial plurality of images or frames based on the pre-processed first layer reconstructed images or frames and for decomposing a checkerboard interleaved frame into upconverted component planes,
wherein all of the two or more views are decoded and processed in each of the first enhancement layer and the second enhancement layer.
7. The apparatus of claim 6 , wherein the two or more views are interleaved in a checkerboard pattern.
8. A non-transitory computer-readable medium containing a set of instructions that causes a computer to perform a process for decoding a two-dimensional (2D) compatible or frame compatible three-dimensional (3D), wherein the process comprises the steps of:
first enhancement layer video decoding an initial plurality of encoded images or frames, each image or frame comprising two or more views, to form first enhancement layer reconstructed images or frames;
storing, in an first enhancement layer decoding reference buffer, the first enhancement layer reconstructed images or frames;
pre-processing first enhancement layer video reconstructed images or frames;
storing, in a second enhancement layer decoding reference buffer, the pre-processed first enhancement layer reconstructed images or frames for second enhancement layer video decoding;
second enhancement layer video decoding the initial plurality of images or frames based on the pre-processed first enhancement layer reconstructed images or frames; and,
decomposing a checkerboard interleaved frame into upconverted component planes,
wherein all views of the initial plurality of encoded images or frames are decoded and processed in each of the first enhancement layer and the second enhancement layer.
9. The non-transitory computer-readable medium of claim 8 , wherein the two or more views are interleaved in a checkerboard pattern.