Systems and methods for dynamic allocation of computing resources for microservice architecture type applications
Systems and methods for auto-scaling computing resources for applications that rely on microservices, such as media streaming applications. In example embodiments, the systems and methods use a hybrid approach that takes into account proactive and reactive metrics to predict future processing needs. The proactive metrics use data obtained from the microservices pipeline. In example embodiments, computing units are deployed or removed from a cluster of computing units in anticipation of a future computational need and in consideration of quality of service requirements.
1. A method of scaling a number of computer processing units, comprising the steps of:
receiving processing requests, each processing request requiring a first processing service and a subsequent second processing service that receives output from the first service, processes the output and completes the processing request;
monitoring, for each request, at each of a plurality of times that precede a predefined time, a plurality of previous processing events in an event sequence of the first processing service that processes the request, each of the previous processing events having an associated action type that is different from an associated action type of at least one later action performed by the second processing service that further processes the output of the first processing service for at least one of the requests, to provide previous events data;
selecting a number of computer processing units to perform the at least one later action of the second processing service at the predefined time, the selecting being based on the previous events data; and
making available at the predefined time the selected number of computer processing units to perform the at least one later action of the second processing service to complete the at least one of the requests.
2. The method of claim 1 , wherein the previous events data includes the associated action type of each previous event and a number of instances of each of the previous events occurring at each of the times that precede the predefined time.
3. The method of claim 1 , wherein each of the previous events is associated with one of a plurality of microservices of the first processing service.
4. The method of claim 3 ,
wherein the action types include a parsing type and file downloading type; and
wherein the previous events data include:
when a request has been parsed;
when a file download has begun; and
when a file download has finished.
5. The method of claim 3 , wherein each event sequence and the selecting are triggered by an availability of at least one new or modified media content item.
6. The method of claim 1 ,
wherein the selecting the number of computer processing units is also based on one or more of:
(i) a current length of a queue of the processing requests;
(ii) a predetermined processing throughput of each of the computer processing units; and
(iii) a predefined maximum length of time to complete processing of the queue.
7. The method of claim 6 ,
wherein the first processing service includes a request ingestion service;
wherein the second processing service includes a transcoding service that receives and processes output of the request ingestion service; and
wherein each of the event sequences is triggered by a request to publish at least one media content item or by a request to stream at least one media content item received by the request ingestion service.
8. The method of claim 7 ,
wherein the computer processing units are made available for the transcoding service; and
wherein the at least one later action includes a transcoding of a media content item for streaming.
9. The method of claim 7 , wherein the previous events data include:
when a request has been parsed by the request ingestion service;
when a file download has been begun by the request ingestion service;
when a file download has been finished by the request ingestion service; and
when the request has been received by the transcoder service.
10. The method of claim 1 ,
wherein the previous events data is weighted based on:
the associated time that precedes the predefined time; and
the associated action type.
11. The method of claim 1 , further comprising: performing, at the predefined time, the at least one later action with the made available computer processing units.
12. A system for scaling a number of computer processing units, comprising:
one or more processors and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to:
receive processing requests, each processing request requiring a first processing service and a subsequent second processing service that receives output from the first service, processes the output and completes the processing request;
monitor, for each request, at each of a plurality of times that precede a predefined time, a plurality of previous processing events in an event sequence of the first processing service that processes the request, each of the previous processing events having an associated action type that is different from an associated action type of at least one later action performed by the second processing service that further processes the output of the first processing service for at least one of the requests, to provide previous events data;
select a number of computer processing units to perform the at least one later action of the second processing service at the predefined time, to select being based on the previous events data; and
make available at the predefined time the selected number of computer processing units to perform the at least one later action of the second processing service to complete the at least one of the requests.
13. The system of claim 12 , wherein the previous events data includes the associated action type of each previous event and a number of instances of each of the previous events occurring at each of the times that precede the predefined time.
14. The system of claim 12 , wherein each of the previous events is associated with one of a plurality of microservices of the first processing service.
15. The system of claim 14 ,
wherein the action types include a parse type and file download type; and
wherein the previous events data include:
when a request has been parsed;
when a file download has begun; and
when a file download has finished.
16. The system of claim 14 , wherein each event sequence is triggered by an availability of at least one new or modified media content item.
17. The system of claim 14 ,
wherein the select the number of computer processing units is also based on one or more of:
(i) a current length of a queue of the processing requests;
(ii) a predetermined processing throughput of each of the computer processing units; and
(iii) a predefined maximum length of time to complete processing of the queue.
18. The system of claim 17 ,
wherein the instructions are operable, when executed by the one or more processors, to cause the one or more processors to provide a request ingestion service and a transcoding service;
wherein each event sequence is associated with the request ingestion service;
wherein the transcoding service receives the processing requests from the request ingestion service; and
wherein each of the event sequences is triggered by a request to publish at least one media content item or by a request to stream at least one media content item received by the request ingestion service.
19. The system of claim 18 ,
wherein the computer processing units are made available for the transcoding service; and
wherein the at least one later action includes a transcoding of a media content item for streaming.
20. The system of claim 18 , wherein the previous events data include:
when a request has been parsed by the request ingestion service;
when a file download has been begun by the request ingestion service;
when a file download has been finished by the request ingestion service; and
when the request has been received by the transcoder service.
21. The system of claim 12 ,
wherein the previous events data is weighted based on:
the associated time that precedes the predefined time; and
the associated action type.
22. The system of claim 12 , wherein the instructions, when executed by the one or more processors, further cause the one or more made available processing units to perform, at the predefined time, the at least one later action.
23. A non-transitory computer-readable medium having stored thereon one or more sequence of instructions for causing one or more processors to perform:
receiving processing requests, each processing request requiring a first processing service and a subsequent second processing service that receives output from the first service, processes the output and completes the processing request;
monitoring, for each request, at each of a plurality of times that precede a predefined time, a plurality of previous processing events in an event sequence of the first processing service that processes the request, each of the previous processing events having an associated action type that is different from an associated action type of at least one later action performed by the second processing service that further processes the output of the first processing service for at least one of the requests, to provide previous events data;
selecting a number of computer processing units to perform the at least one later action of the second processing service at the predefined time, the selecting being based on the previous events data; and
making available at the predefined time the selected number of computer processing units to perform the at least one later action of the second processing service to complete the at least one of the requests.
24. The non-transitory computer-readable medium of claim 23 , wherein the previous events data includes the associated action type of each previous event and a number of instances of each of the previous events occurring at each of the times that precede the predefined time.
25. The non-transitory computer-readable medium of claim 23 , wherein each of the previous events is associated with one of a plurality of microservices of the first processing service.
26. The non-transitory computer-readable medium of claim 25 ,
wherein the action types include a parsing type and file downloading type; and
wherein the previous events data include:
when a request has been parsed;
when a file download has begun; and
when a file download has finished.
27. The non-transitory computer-readable medium of claim 25 , wherein each event sequence is triggered by an availability of at least one new or modified media content item.
28. The non-transitory computer-readable medium of claim 23 ,
wherein the selecting the number of computer processing units is also based on one or more of:
(i) a current length of a queue of the processing requests;
(ii) a predetermined processing throughput of each of the computer processing units; and
(iii) a predefined maximum length of time to complete processing of the queue.
29. The non-transitory computer-readable medium of claim 28 ,
wherein each event sequence is associated with a request ingestion service;
wherein a transcoding service receives the processing requests from the request ingestion service; and
wherein each of the event sequences is triggered by a request to publish at least one media content item or by a request to stream at least one media content item received by the request ingestion service.
30. The non-transitory computer-readable medium of claim 29 ,
wherein the computer processing units are made available for the transcoding service; and
wherein the at least one later action includes a transcoding of a media content item for streaming.
31. The non-transitory computer-readable medium of claim 29 , wherein the previous events data include:
when a request has been parsed by the request ingestion service;
when a file download has been begun by the request ingestion service;
when a file download has been finished by the request ingestion service; and
when the request has been received by the transcoder service.
32. The non-transitory computer-readable medium of claim 23 ,
wherein the previous events data is weighted based on:
the associated time that precedes the predefined time; and
the associated action type.
33. The non-transitory computer-readable medium of claim 23 , wherein the one or more sequences of instructions cause the made available computer processing units to perform, at the predefined time, the at least one later action.