Systems and methods for encoding and playing back video at different frame rates using enhancement layers
Systems and methods in accordance with embodiments of this invention provide for encoding and playing back video at different frame rates using enhancement layers. In a number of embodiments, video is encoded using temporal scalability involving the creation of a base layer at a first frame rate and an enhancement layer including additional frames enabling playback at a second higher frame rate. The second higher frame rate can also be referred to as an enhanced frame rate. In a number of embodiments, the base and enhancement layers are stored in one or more container files that contain metadata describing the enhancement layer. Based on the capabilities of a playback device, it can select the particular frame rate at which to playback encoded video.
1. A method of encoding video from a source video, comprising:
receive video having a plurality of access units ordered in a decode order using at least one source encoding server system, wherein the decode order is an order that causes a decoder to have fully decoded all frames on which a particular frame depends prior to decoding the particular frame;
encode the video into a set of layers comprising a base layer and at least one enhancement layer, wherein the encoding comprises: (i) using a temporal identifier of each access unit to determine a particular layer associated with the access unit and (ii) retaining as metadata the decode order of the temporal identifiers of the plurality of access units of the video using the at least one source encoding server system; and
storing the set of layers into separate container files, wherein the enhancement layer is stored in a separate container file that also includes the metadata with the decode order of the temporal identifiers of the plurality of access units of the video;
wherein the base layer comprises a sequence of frames encoded at a first frame rate such that frames in the base layer only depend upon other frames in the base layer;
wherein the enhancement layer comprises additional frames that, when merged with the sequence of frames in the base layer, form a sequence of frames encoded at a second frame rate, and frames in the enhancement layer are encoded such that the frames in the enhancement layer only depend upon frames in the base layer; and
wherein the enhancement layer comprises the metadata that is used to determine the decode order and that specifies the decode order for selecting access units from the base layer and the enhancement layer to combine the plurality of access units into a single video stream in the decode order.
2. The method of claim 1 , wherein a temporal identifier of an access unit is stored in an initial set of bits of the access unit.
3. The method of claim 1 , further comprising inserting a temporal identifier for a set of access units of the plurality of access units of the video.
4. The method of claim 1 , further comprising inserting a metadata tag providing information regarding the layers available and the frame rate of each layer.
5. The method of claim 1 , further comprising:
receiving a request for streaming of the video;
determining playback capabilities of a device requesting the video; and
streaming different sets of layers of the video based on the playback capabilities of the device, wherein (i) the base layer is streamed to a playback device capable of playing back the base layer and (ii) the base layer and at least one enhancement layer is streamed to a playback device capable of streaming the plurality of layers.
6. The method of claim 5 , wherein streaming the different sets of layers comprises merging the different layers using the retained order of the temporal identifiers of the plurality of access units of the video.
7. The method of claim 1 , further comprising:
receiving a request for downloading of the video from a playback device; and
providing the encoded video to the playback device.
8. The method of claim 1 , wherein the frame rate of the at least one enhancement layer is an integer multiple of the base layer.
9. The method of claim 1 , further comprising storing each layer as a separate stream.