System and method of providing cloud bursting capabilities in a compute environment using templates
Technologies are provided for a multi-cloud bursting service. The service generates a template for provisioning a compute environment in a selected cloud service provider. The template defines a software application to be used for a job to be processed by the compute environment and a hardware setting identifying hardware to be used in the compute environment. The service receives, from a local compute environment, a request to provision the compute environment and transmits the job and the template to the selected cloud service provider such that the selected cloud service provider, based on the cloud agnostic burst template: provisions the compute environment; and deploys the job on the compute environment in the selected cloud service provider.
1. A method comprising:
generating a cloud agnostic burst template for provisioning a compute environment in a selected cloud service provider available from a plurality of different cloud service providers, the cloud agnostic burst template comprising a stack template and a job template, wherein the stack template defines: a software application to be used for a job to be processed by the compute environment; and a hardware setting identifying hardware to be used in the compute environment, and wherein the job template specifies that the job is meant to be run at a selected cloud service provider from the plurality of different cloud service providers independent of a state of a job backlog;
receiving, at a multi-cloud bursting service, from a local compute environment, the job and a request to provision the compute environment in one or more of the plurality of different cloud service providers; and
transmitting, from the multi-cloud bursting service, the job and the cloud agnostic burst template to the selected cloud service provider such that the selected cloud service provider, based on the cloud agnostic burst template:
provisions the compute environment including the software application; and
deploys the job on the compute environment to be processed by the software application.
2. The method of claim 1 , wherein the cloud agnostic burst template further defines one or more commands associated with enabling the job to be processed in the compute environment.
3. The method of claim 1 , wherein the cloud agnostic burst template further defines a number of cores to use in the compute environment.
4. The method of claim 1 , wherein the cloud agnostic burst template further defines a parameter associated with time in connection with use of the compute environment.
5. The method of claim 1 , wherein the multi-cloud bursting service provides a status of provisioning and/or using the compute environment in the selected cloud service provider.
6. The method of claim 1 , wherein the cloud agnostic burst template further defines a license associated with use of the compute environment.
7. The method of claim 2 , wherein the selected cloud service provider further:
launches, on the compute environment, at least one instance of a chosen workload environment associated with the cloud agnostic burst template; and
assigns the job to the at least one instance of the chosen workload environment.
8. The method of claim 1 , wherein the selected cloud service provider further provisions the compute environment by translating one or more commands associated with the cloud agnostic burst template into one or more cloud provider commands defined for the selected cloud service provider based at least partly on at least one of a configuration or a characteristic of the selected cloud service provider, the one or more commands being executable for provisioning the compute environment and deploying a workload environment at the selected cloud service provider.
9. The method of claim 7 , wherein the one or more commands comprise executable instructions for at least one of: uploading one or more files associated with the cloud agnostic burst template to the compute environment, requesting a provisioning of the compute environment, installing the software application, and configuring the software application.
10. The method of claim 2 , wherein the cloud agnostic burst template is generated by a user or stored in a database of cloud agnostic burst templates available for selection by the user.
11. The method of claim 9 , wherein the cloud agnostic burst template is associated with one or more job definitions, the one or more job definitions specifying, for each job, at least one of a respective job identifier, a stack identifier corresponding to a stack for the job, and one or more parameters associated with the job.
12. The method of claim 11 , wherein the one or more parameters specify at least one of a trigger for provisioning the compute environment for the job, a type of cloud instance for the job, a number of cloud instances for the job, a number of cloud nodes to request for the job, a number of processors for each cloud node, and one or more job policies.
13. The method of claim 12 , wherein the one or more job policies comprise at least one of a quality of service, a node or resource usage limit, a cloud bursting limit, and a purge condition defining at least one of a time-to-live and a node idle purge time, the trigger comprising at least one of a threshold backlog, a threshold resource availability, a policy violation, and a threshold condition.
14. A system comprising:
one or more processors; and
at least one non-transitory computer-readable storage medium storing instructions which, when executed by the one or more processors, cause the system to:
generate a cloud agnostic burst template for provisioning a compute environment in a selected cloud service provider available from a plurality of different cloud service providers, the cloud agnostic burst template comprising a stack template and a job template, wherein the stack template defines: a software application to be used for a job to be processed by the compute environment; and a hardware setting identifying hardware to be used in the compute environment, and wherein the job template specifies that the job is meant to be run at a selected cloud service provider from the plurality of different cloud service providers independent of a state of a job backlog;
receive, from a local compute environment, the job and a request to provision the compute environment in one or more of the plurality of different cloud service providers; and
transmit the job and the cloud agnostic burst template to the selected cloud service provider such that the selected cloud service provider, based on the cloud agnostic burst template:
provisions the compute environment including the software application; and
deploys the job on the compute environment to be processed by the software application.
15. The system of claim 14 , wherein the cloud agnostic burst template further defines one or more: commands associated with enabling the job to be processed in the compute environment; a number of cores to use in the compute environment; a parameter associated with time in connection with use of the compute environment; and a license associated with use of the compute environment.
16. The system of claim 14 , wherein the selected cloud service provider further:
launches, on the compute environment, at least one instance of a chosen workload environment associated with the cloud agnostic burst template; and
assigns the job to the at least one instance of the chosen workload environment.
17. The system of claim 16 , wherein the selected cloud service provider further provisions the compute environment by translating one or more commands associated with the cloud agnostic burst template into one or more cloud provider commands defined for the selected cloud service provider based at least partly on at least one of a configuration or a characteristic of the selected cloud service provider, the one or more commands being executable for provisioning the compute environment and deploying a workload environment at the selected cloud service provider.
18. The system of claim 17 , wherein the one or more commands comprise executable instructions for at least one of: uploading one or more files associated with the cloud agnostic burst template to the compute environment, requesting a provisioning of the compute environment, installing the software application, and configuring the software application.
19. The system of claim 14 , wherein the cloud agnostic burst template is generated by a user or stored in a database of cloud agnostic burst templates available for selection by the user.
20. The system of claim 19 , wherein the cloud agnostic burst template is associated with one or more job definitions, the one or more job definitions specifying, for each job, at least one of a respective job identifier, a stack identifier corresponding to a stack for the job, and one or more parameters associated with the job.
21. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to:
generate a cloud agnostic burst template for provisioning a compute environment in a selected cloud service provider available from a plurality of different cloud service providers, the cloud agnostic burst template comprising a stack template and a job template, wherein the stack template defines: a software application to be used for a job to be processed by the compute environment; and a hardware setting identifying hardware to be used in the compute environment, and wherein the job template specifies that the job is meant to be run at a selected cloud service provider from the plurality of different cloud service providers independent of a state of a job backlog;
receive, from a local compute environment, the job and a request to provision the compute environment in one or more of the plurality of different cloud service providers; and
transmit the job and the cloud agnostic burst template to the selected cloud service provider such that the selected cloud service provider, based on the cloud agnostic burst template:
provisions the compute environment including the software application; and
deploys the job on the compute environment to be processed by the software application.