Systems and methods for frame duplication and frame extension in live video encoding and streaming
Embodiments of the invention provide for live encoding systems that can replicate a current encoded frame instead of re-encoding said current frame, and then adjust the replicated frame to different bitrates, resolutions, and/or contexts as necessary for the several different adaptive bitrate streams. In addition, various embodiments of the invention can extend a duration of a current frame being repackaged and/or re-encoded. Utilizing these and other techniques, live encoding systems in accordance with embodiments of the invention can more efficiently handle gaps in received data, slower feeding of data, and/or heavy loads on server hardware.
1. A method of encoding an input stream into a plurality of adaptive bitrate streams using a live encoding system, the method comprising:
receiving an input stream using a live encoding system;
assessing encoding conditions using the live encoding system; and
encoding a given segment of the input stream into a plurality of adaptive bitrate segments using the live encoding system, wherein encoding the given segment of the input stream into the plurality of adaptive bitrate segments comprises:
replicating at least one frame from the given segment of the input stream and using the replicated at least one frame from the given segment of the input stream in at least one of the plurality of adaptive bitrate segments when the assessed encoding conditions satisfy a first threshold; and
encoding frames of the given segment of the input stream into the plurality of adaptive bitrate segments using the live encoding system when the assessed encoding conditions do not satisfy the first threshold.
2. The method of claim 1 , wherein assessing the encoding conditions for later segments comprises determining an arrival time between a previous segment and the given segment, wherein encoding is performed when an assessed server load level exceeds a threshold, wherein the threshold is a particular amount of time between arrivals of different segments.
3. The method of claim 1 , further comprising streaming the encoded plurality of adaptive bitrate segments to a plurality of streaming clients over at least one network.
4. The method of claim 1 , wherein each adaptive bitrate segment of the plurality of adaptive bitrate segments are encoded into different maximum bitrates.
5. The method of claim 1 , wherein the input stream is a live stream selected from the group comprising sporting events, live news coverage, web live streams, singular streams of media and multiplexed streams of media.
6. The method of claim 1 , further comprising assessing encoding conditions using the live encoding system, wherein replicating the at least one frame from the given segment of the input stream and encoding frames of the given segment of the input stream into the plurality of adaptive bitrate segments are performed based on the assessed encoding conditions.
7. The method of claim 1 , wherein the input stream is a live stream.
8. The method of claim 2 , wherein encoding the given segment of the input stream into the plurality of adaptive bitrate segments comprises:
extending at least one frame, and adjusting timestamps accordingly, from the given segment of the input stream by increasing a time duration of the at least one frame; and
using the extended at least one frame from the given segment of the input stream in at least one of the plurality of adaptive bitrate segments when the arrival time between the previous segment and the given segment of the input stream satisfy a second threshold.
9. The method of claim 2 , wherein determining the arrival time between the previous segment and the given segment further comprises comparing time stamp differences between the previous segment and the given segment to a clock implementation on the live encoding system.
10. The method of claim 2 , wherein assessing encoding conditions includes sub-operations comprising:
checking frame rates of the received input stream;
comparing the frame rates of the received input stream to frame times determined internally by the live encoding system;
measuring time differences between incoming frames to calculate an overall incoming data rate;
monitoring the overall incoming data rate to identify gaps in incoming data; and
predicting whether the live encoding system has not received a frame at a proper time.
11. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out a method that comprises:
receiving an input stream using a live encoding system;
assessing encoding conditions using the live encoding system; and
encoding a given segment of the input stream into a plurality of adaptive bitrate segments using the live encoding system, wherein encoding the given segment of the input stream into the plurality of adaptive bitrate segments comprises:
replicating at least one frame from the given segment of the input stream and using the replicated at least one frame from the given segment of the input stream in at least one of the plurality of adaptive bitrate segments when the assessed encoding conditions satisfy a first threshold; and
encoding frames of the given segment of the input stream into the plurality of adaptive bitrate segments using the live encoding system when the assessed encoding conditions do not satisfy the first threshold.
12. The computer-readable medium of claim 11 , wherein assessing the encoding conditions for later segments comprises determining an arrival time between a previous segment and the given segment, wherein encoding is performed when an assessed server load level exceeds a threshold, wherein the threshold is a particular amount of time between arrivals of different segments.
13. The computer-readable medium of claim 11 , further comprising streaming the encoded plurality of adaptive bitrate segments to a plurality of streaming clients over at least one network.
14. The computer-readable medium of claim 11 , wherein each adaptive bitrate segment of the plurality of adaptive bitrate segments are encoded into different maximum bitrates.
15. The computer-readable medium of claim 11 , wherein the input stream is a live stream selected from the group comprising sporting events, live news coverage, web live streams, singular streams of media and multiplexed streams of media.
16. The computer-readable medium of claim 11 , further comprising assessing encoding conditions using the live encoding system, wherein replicating the at least one frame from the given segment of the input stream and encoding frames of the given segment of the input stream into the plurality of adaptive bitrate segments are performed based on the assessed encoding conditions.
17. The computer-readable medium of claim 11 , wherein the input stream is a live stream.
18. The computer-readable medium of claim 12 , wherein encoding the given segment of the input stream into the plurality of adaptive bitrate segments comprises:
extending at least one frame, and adjusting timestamps accordingly, from the given segment of the input stream by increasing a time duration of the at least one frame; and
using the extended at least one frame from the given segment of the input stream in at least one of the plurality of adaptive bitrate segments when the arrival time between the previous segment and the given segment of the input stream satisfy a second threshold.
19. The computer-readable medium of claim 12 , wherein determining the arrival time between the previous segment and the given segment further comprises comparing time stamp differences between the previous segment and the given segment to a clock implementation on the live encoding system.
20. The computer-readable medium of claim 12 , wherein assessing encoding conditions includes sub-operations comprising:
checking frame rates of the received input stream;
comparing the frame rates of the received input stream to frame times determined internally by the live encoding system;
measuring time differences between incoming frames to calculate an overall incoming data rate;
monitoring the overall incoming data rate to identify gaps in incoming data; and
predicting whether the live encoding system has not received a frame at a proper time.