HIERARCHICAL VIDEO COMPRESSION SUPPORTING SELECTIVE DELIVERY OF TWO-DIMENSIONAL AND THREE-DIMENSIONAL VIDEO CONTENT
Systems, methods and apparatuses are described herein for encoding a plurality of video frame sequences, wherein each video frame sequence corresponds to a different perspective view of the same subject matter. In accordance with various embodiments, the encoding is performed in a hierarchical manner that leverages referencing between frames of different ones of the video frame sequences (so-called “external referencing”), but that also allows for encoded representations of only a subset of the video frame sequences to be provided when less than all sequences are required to support a particular viewing mode of a display system that is capable of displaying the subject matter in a two-dimensional mode, a three-dimensional mode, or a multi-view three-dimensional mode.
1 . A method for encoding a first video frame sequence and a second video frame sequence, the first video frame sequence corresponding to a first perspective view, the second frame sequence corresponding to a second perspective view, the method comprising:
applying a first tier of hierarchical encoding to the first video frame sequence to produce first tier encoded data, the first tier of hierarchical encoding using first internal referencing, the first internal referencing involving multiple frames within the first video frame sequence;
applying a second tier of hierarchical encoding to the second video frame sequence to produce second tier encoded data, the second tier of hierarchical encoding using both second internal referencing and first external referencing, the second internal referencing involving multiple frames within the second video frame sequence, the first external referencing involving references between a frame of the second video frame sequence and a frame of the first video frame sequence; and
the first tier encoded data being independently decodable, and the second tier encoded data being decodable with access to the first tier encoded data.
2 . The method of claim 1 , further comprising delivering the first tier encoded data to service a two-dimensional viewing environment.
3 . The method of claim 1 , further comprising delivering both the first tier encoded data and the second tier encoded data to service a three-dimensional viewing environment.
4 . The method of claim 1 , further comprising applying a third tier of hierarchical encoding to a third video frame sequence corresponding to a third perspective view, the third tier of hierarchical encoding using both third internal referencing and second external referencing, the third internal referencing involving multiple frames within the third video frame sequence, the second external referencing involving references between a frame of the third video frame sequence and a frame of the second video frame sequence.
5 . The method of claim 1 , further comprising applying a third tier of hierarchical encoding to a third video frame sequence corresponding to a third perspective view, the third tier of hierarchical encoding using both third internal referencing and second external referencing, the third internal referencing involving multiple frames within the third video frame sequence, the second external referencing involving references between a frame of the third video frame sequence and a frame of the first video frame sequence.
6 . The method of claim 1 , further comprising applying a third tier of hierarchical encoding to a third video frame sequence corresponding to a third perspective view, the third tier of hierarchical encoding using both third internal referencing and second external referencing, the third internal referencing involving multiple frames within the third video frame sequence, the second external referencing involving references between a first frame of the third video frame sequence and a frame of the first video frame sequence and references between a second frame of the third video frame sequence and a frame of the second video frame sequence.
7 . A method used to encode a plurality of video elements, each of the plurality of video elements representing a video frame sequence from a selected perspective view, the method comprising:
identifying a first of the plurality of video elements, the first of the plurality of video elements having a first video frame sequence corresponding to a first perspective view;
producing an encoded version of the first video frame sequence, the encoded version of the first video frame sequence being independently decodable;
identifying a second of the plurality of video elements, the second of the plurality of video elements having a second video frame sequence corresponding to a second perspective view; and
producing an encoded version of the second video frame sequence, the encoded version of the second video frame sequence being decodable with reference to at least a representation of a frame of the first video frame sequence.
8 . The method of claim 7 , wherein the encoded version of the first video frame sequence can be decoded to produce a two-dimensional output.
9 . The method of claim 7 , wherein the encoded versions of both the first video frame sequence and the second video frame sequence can be decoded to produce a three-dimensional output.
10 . The method of claim 7 , further comprising producing an encoded version of a third video frame sequence, the encoded version of the third video frame sequence being decodable with reference to at least a representation of a frame of the second video frame sequence.
11 . The method of claim 7 , further comprising producing an encoded version of a third video frame sequence, the encoded version of the third video frame sequence being decodable with reference to at least a representation of a frame of the first video frame sequence.
12 . The method of claim 7 , further comprising producing an encoded version of a third video frame sequence, the encoded version of the third video frame sequence being decodable with reference to at least a representation of a frame of the first video frame sequence and a representation of a frame of the second video frame sequence.
13 . An encoding system that receives video content, the video content having both first video data representative of a first perspective view of three-dimensional content and second video data representative of a second perspective view of three-dimensional content, the encoding system comprising:
a processing circuit that, using a first predefined encoding approach, operates on the first video data to produce a first encoded portion of an encoded version of the video content;
the processing circuit that, using a second predefined encoding approach, operates on the second video data to produce a second encoded portion of the encoded version of the video content, the second predefined encoding approach differing from the first predefined encoding approach in at least that the second predefined encoding approach supports external referencing; and
output circuitry, coupled to the processing circuitry, through which the first encoded portion and the second encoded portion are communicated.
14 . The encoding system of claim 13 , wherein the first predefined encoding approach comprises first tier hierarchical encoding and the second predefined encoding approach comprises second tier hierarchical encoding.
15 . The encoding system of claim 13 , wherein the video content also has third video data representative of a third perspective view of three-dimensional content;
wherein the second predefined encoding approach supports external referencing between frames of the second video data and frames of the first video data;
wherein the processing circuit, using a third predefined encoding approach, operates on the third video data to produce a third encoded portion of the encoded version of the video content, the third predefined encoding approach supporting external referencing between frames of the third video data and frames of the second video data and external referencing between frames of the third video data and frames of the first video data.
16 . The encoding system of claim 15 , wherein the first predefined encoding approach comprises first tier hierarchical encoding, the second predefined encoding approach comprises second tier hierarchical encoding, and the third predefined encoding approach comprises third tier hierarchical encoding.
17 . A decoding system that receives encoded video content, the encoded video content having both a first encoded portion relating to a first perspective view of three-dimensional content and a second encoded portion relating to a second perspective view of three-dimensional content, the decoding system comprising:
a processing circuit that, using a first predefined decoding approach, operates on the first encoded portion of the encoded video content to produce first video data, the first video data representative of the first perspective view of the three-dimensional content;
the processing circuit that, using a second predefined decoding approach, operates on the second encoded portion of the encoded video content to produce second video data, the second video data representative of the second perspective view of the three-dimensional content, and the second predefined decoding approach differing from the first predefined decoding approach in at least that the second predefined decoding approach supports external referencing; and
output circuitry, coupled to the processing circuitry, through which the first video data and the second video data are communicated.
18 . The decoding system of claim 11 , wherein the first predefined decoding approach comprises first tier hierarchical decoding and the second predefined decoding approach comprises second tier hierarchical decoding.
19 . The decoding system of claim 11 , wherein the encoded video content also has a third encoded portion relating to a third perspective view of three-dimensional content;
wherein the second predefined decoding approach supports external referencing between frames represented by the second encoded portion of the encoded video content and frames represented by the first encoded portion of the video content;
wherein the processing circuit, using a third predefined decoding approach, operates on the third encoded portion of the encoded video content to produce third video data, the third video data representative of the third perspective view of the three-dimensional content, the third predefined decoding approach supporting external referencing between frames represented by the third encoded portion of the encoded video content and frames represented by the second encoded portion of the encoded video content and external referencing between frames represented by the third encoded portion of the encoded video content and frames represented by the first encoded portion of the encoded video content.
20 . The decoding system of claim 13 , wherein the first predefined decoding approach comprises first tier hierarchical decoding, the second predefined decoding approach comprises second tier hierarchical decoding, and the third predefined decoding approach comprises third tier hierarchical decoding.