Distributed scalable encoder resources for live streams
Live media streams are delineated for distributed encoding and fragmentation in a dynamically scalable distributed resource system. In some examples, live MPEG-2 media streams are separated into groups of pictures (GOPs) and converted into jobs for encoding and fragmentation systems. Multiple jobs may be created for the same live media stream to provide for different quality levels for each fragment of each channel. Shared resources such as a pool of dynamically scalable virtual machines can be used to process the individual jobs. Encoded fragments may be H.264 fragments maintained on shared storage. A particular live stream corresponding to a particular quality level can be reconstructed using the encoded fragments.
1. A method, comprising:
placing a plurality of jobs associated with a first group of pictures (GOP) associated with a media stream into a plurality of job queues including an active job queue and a standby job queue, the plurality of job queues accessible by a plurality of encoders and fragmentation systems, wherein additional encoders and fragmentation systems are activated when resource usage in the plurality of encoders and fragmentation systems reaches a particular threshold and wherein the additional encoders and fragmentation systems are deactivated when resource usage falls below a particular threshold, wherein at least one of the plurality of jobs is placed in the standby job queue and at least one of the plurality of jobs is placed in the active job queue, each job queue being a first in first out queue;
performing encoding and fragmentation, prior to transmission to a user device, by the plurality of encoders and fragmentation systems to produce a plurality of encoded fragments, wherein the performing encoding and fragmentation is initiated prior to receiving a complete file header associated with the media stream, and wherein each of the plurality of encoded fragments comprises a box structure including: synchronization information comprising audio and video synchronization information, end of file information, and chapter information;
transmitting, by the plurality of encoders and fragmentation system, the plurality of encoded fragments to distributed storage; and
maintaining the plurality of encoded fragments in distributed storage.
2. The method of claim 1 , wherein a GOP delineator resides in a first data center.
3. The method of claim 2 , wherein the plurality of encoders and fragmentation systems reside in a second data center.
4. The method of claim 3 , wherein the first GOP is encrypted prior to transmission to the second data center.
5. The method of claim 1 , wherein the plurality of encoders and fragmentation systems are dynamically scalable virtual machines.
6. The method of claim 5 , wherein an additional virtual machine is added to the plurality of virtual machines when processor resource usage exceeds a particular threshold.
7. The method of claim 5 , wherein an additional virtual machine is added to the plurality of virtual machines when the job queue level exceeds a particular threshold.
8. The method of claim 5 , wherein an additional virtual machine is added to the plurality of virtual machines when encoding delay exceeds a particular threshold.
9. The method of claim 1 , wherein the plurality of encoders and fragmentation systems is implemented using a plurality of virtual machines in a cloud environment.
10. The method of claim 1 , wherein the plurality of encoded fragments is a plurality of MPEG-4 fragments.
11. A system, comprising:
a processor; and
memory, the memory configured to store instructions to cause the processor to execute a method, the method comprising:
placing a plurality of jobs associated with a first group of pictures (GOP) associated with a media stream into a plurality of job queues including an active job queue and a standby job queue, the plurality of job queues accessible by a plurality of encoders and fragmentation systems, wherein additional encoders and fragmentation systems are activated when resource usage in the plurality of encoders and fragmentation systems reaches a particular threshold and wherein the additional encoders and fragmentation systems are deactivated when resource usage falls below a particular threshold, wherein at least one of the plurality of jobs is placed in the standby job queue and at least one of the plurality of jobs is placed in the active job queue, each job queue being a first in first out queue;
performing encoding and fragmentation, prior to transmission to a user device, by the plurality of encoders and fragmentation systems to produce a plurality of encoded fragments, wherein the performing encoding and fragmentation is initiated prior to receiving a complete file header associated with the media stream, and wherein each of the plurality of encoded fragments comprises a box structure including: synchronization information comprising audio and video synchronization information, end of file information, and chapter information;
transmitting, by the plurality of encoders and fragmentation system, the plurality of encoded fragments to distributed storage; and
maintaining the plurality of encoded fragments in distributed storage.
12. The system of claim 11 , wherein a GOP delineator resides in a first data center.
13. The system of claim 12 , wherein the plurality of encoders and fragmentation systems reside in a second data center.
14. The system of claim 13 , wherein the first GOP is encrypted prior to transmission to the second data center.
15. The system of claim 11 , wherein the plurality of encoders and fragmentation systems are dynamically scalable virtual machines.
16. The system of claim 15 , wherein an additional virtual machine is added to the plurality of virtual machines when processor resource usage exceeds a particular threshold.
17. The system of claim 15 , wherein an additional virtual machine is added to the plurality of virtual machines when the job queue level exceeds a particular threshold.
18. The system of claim 15 , wherein an additional virtual machine is added to the plurality of virtual machines when encoding delay exceeds a particular threshold.
19. A non-transitory computer readable medium storing instructions to execute a method, the method comprising:
placing a plurality of jobs associated with a first group of pictures (GOP) associated with a media stream into a plurality of job queues, the plurality of job queues accessible bya plurality of encoders and fragmentation systems, wherein additional encoders and fragmentation systems are activated when resource usage in the plurality of encoders and fragmentation systems reaches a particular threshold and wherein the additional encoders and fragmentation systems are deactivated when resource usage falls below a particular threshold, wherein the plurality of job queues includes an active job queue and a standby job queue, each job queue being a first in first out queue;
performing encoding and fragmentation, prior to transmission to a user device, by the plurality of encoders and fragmentation systems to produce a plurality of encoded fragments, wherein the performing encoding and fragmentation is initiated prior to receiving a complete file header associated with the media stream, and wherein each of the plurality of encoded fragments comprises a box structure including: synchronization information comprising audio and video synchronization information, end of file information, and chapter information;
transmitting, by the plurality of encoders and fragmentation system, the plurality of encoded fragments to distributed storage; and
maintaining the plurality of encoded fragments in distributed storage.