Systems and methods for frame duplication and frame extension in live video encoding and streaming
View Patent ↗Embodiments of the invention provide for live encoding systems that can replicate a current encoded frame instead of re-encoding said current frame, and then adjust the replicated frame to different bitrates, resolutions, and/or contexts as necessary for the several different adaptive bitrate streams. In addition, various embodiments of the invention can extend a duration of a current frame being repackaged and/or re-encoded. Utilizing these and other techniques, live encoding systems in accordance with embodiments of the invention can more efficiently handle gaps in received data, slower feeding of data, and/or heavy loads on server hardware.
1. A method of encoding an encoded input stream into a plurality of adaptive bitrate streams using a live encoding system, the method comprising:
receiving an encoded input stream using a live encoding system;
assessing encoding conditions using the live encoding system, wherein assessing encoding conditions using the live encoding system further comprises calculating an amount of load on the live encoding system using the live encoding system; and
re-encoding a given segment of the encoded input stream into a plurality of adaptive bitrate segments using the live encoding system based on the assessed encoding conditions, wherein each adaptive bitrate segment of the plurality of adaptive bitrate segments is encoded for a different bitrate, wherein re-encoding the given segment of the input stream into the plurality of adaptive bitrate segments comprises:
extending the time duration of at least one frame having a first duration from the given segment of the input stream in at least one of the plurality of adaptive bitrate segments by modifying a time stamp of a subsequent frame in the at least one adaptive bitrate segment such that the at least one frame has an extended second duration in the at least one adaptive bitrate segment when the assessed encoding conditions satisfy a first encoding load threshold;
replicating at least one frame from the given segment of the input stream and using the replicated at least one frame from the segment of the input stream in at least one of the plurality of adaptive bitrate segments when the assessed encoding conditions satisfy a second encoding load threshold; and
re-encoding at least one frame of the given segment of the input stream into the plurality of adaptive bitrate segments using the live encoding system when the assessed encoding conditions do not satisfy either the first or second encoding load thresholds.
2. The method of claim 1 further comprising streaming the encoded plurality of adaptive bitrate segments to a plurality of streaming clients over at least one network.
3. The method of claim 1 , wherein assessing encoding conditions using the live encoding system further comprises determining an arrival time between a previous segment and the given segment, and wherein the first threshold is a particular amount of time between the arrival of different segments.
4. The method of claim 3 , wherein determining the arrival time between the previous segment and the given segment further comprises comparing time stamp differences between the previous segment and the given segment to a clock implementation on the live encoding system.
5. The method of claim 1 , wherein assessing encoding conditions using the live encoding system further comprises calculating an amount of data being received from the input stream using the live encoding system, and wherein the first threshold is a particular quantity of data received over a specified amount of time.
6. The method of claim 1 , wherein the plurality of adaptive bitrate segments are encoded into different maximum bitrates.
7. The method of claim 1 , wherein the input stream is a live stream.
8. The method of claim 7 , wherein the live stream comprises media selected from the group of audio tracks, video tracks, subtitle tracks, and multimedia tracks.
9. The method of claim 1 , wherein a given segment of the encoded input stream has a first encoding, wherein re-encoding at least one frame comprises re-encoding the at least one frame to a different second encoding, and wherein replicating at least one frame from the segment of the input stream comprises replicating the at least one frame with the same first encoding.
10. A live encoding system, the live encoding system comprising:
at least one processing unit;
a memory storing a live encoding application comprising computer instructions, wherein the live encoding application instructs the at least one processing unit to:
receive an encoded input stream;
assess encoding conditions, wherein the instructions to assess encoding conditions further comprises instructions to calculating an amount of load on the live encoding system; and
re-encode a given segment of the encoded input stream into a plurality of adaptive bitrate segments based on the assessed encoding conditions, wherein each adaptive bitrate segment of the plurality of adaptive bitrate segments is encoded for a different bitrate, wherein the instructions to re-encode the given segment of the input stream into the plurality of adaptive bitrate segments further comprise instructions to:
extend the time duration of at least one frame having a first duration from the given segment of the input stream in at least one of the plurality of adaptive bitrate segments by modifying a time stamp of a subsequent frame in the at least one adaptive bitrate segment such that the at least one frame has an extended second duration in the at least one adaptive bitrate segment when the assessed encoding conditions satisfy a first encoding load threshold;
replicate at least one frame from the given segment of the input stream and using the replicated at least one frame from the segment of the input stream in at least one of the plurality of adaptive bitrate segments when the assessed encoding conditions satisfy a second encoding load threshold; and
re-encode at least one frame of the given segment of the input stream into the plurality of adaptive bitrate segments when the assessed encoding conditions do not satisfy either the first or second encoding load thresholds.
11. The live encoding system of claim 10 , wherein the live encoding application further includes instructions to stream the encoded plurality of adaptive bitrate segments to a plurality of streaming clients over at least one network.
12. The live encoding system of claim 10 , wherein the instructions to assess encoding conditions further comprises instructions to determine an arrival time between a previous segment and the given segment, and wherein the first threshold is a particular amount of time between the arrival of different segments.
13. The live encoding system of claim 12 , wherein the instructions to determine the arrival time between the previous segment and the given segment further comprises instructions to compare time stamp differences between the previous segment and the given segment to a clock implementation on the live encoding system.
14. The live encoding system of claim 10 , wherein the instructions to assess encoding conditions further comprises instructions to calculating an amount of data being received from the input stream, and wherein the first threshold is a particular quantity of data received over a specified amount of time.
15. The live encoding system of claim 10 , wherein the plurality of adaptive bitrate segments are encoded into different maximum bitrates.
16. The live encoding system of claim 10 , wherein the input stream is a live stream.
17. The live encoding system of claim 16 , wherein the live stream comprises media selected from the group of audio tracks, video tracks, subtitle tracks, and multimedia tracks.
18. The live encoding system of claim 10 , wherein a given segment of the encoded input stream has a first encoding, wherein re-encoding at least one frame comprises re-encoding the at least one frame to a different second encoding, and wherein replicating at least one frame from the segment of the input stream comprises replicating the at least one frame with the same first encoding.