Building deployment pipelines for a production computing service using live pipeline templates
Techniques are presented for managing a deployment pipeline using an inheritable and extensible source code template—generally referred to as a live pipeline template (LPT). As described, live pipeline templates may be used to manage deployment pipelines which, in turn, are used to launch, maintain, and update the services and systems used to host and provide computing services.
1. A computer-readable storage medium storing instructions, which, when executed on a processor, perform an operation for provisioning a deployment pipeline, the operation comprising:
receiving a definition of an instance of a live pipeline template (LPT), wherein the instance of the LPT is specified using at least a first base pipeline template and a second pipeline template, wherein the first base pipeline template specifies a set of configuration parameters for a plurality of deployment stages of the deployment pipeline, and wherein the second pipeline template extends the first base pipeline template with one or more instance specific parameters for the deployment pipeline;
generating, from the instance of the LPT, an application definition which provides a fully-specified configuration for a plurality of computing services included in the deployment pipeline;
launching a first instance of the deployment pipeline in at least a first cloud computing region by invoking, for each computing service referenced in the application definition, a respective pipeline synthesis driver to configure one of the plurality of computing services in the first cloud computing region according to the fully-specified configuration provided in the application definition; and
launching at least a second instance of the deployment pipeline in at least a second cloud computing region by invoking, for each computing service referenced in the application definition, the respective pipeline synthesis driver to configure one of the plurality of computing services in the second cloud computing region according to the fully-specified configuration provided in the application definition.
2. The computer-readable storage medium of claim 1 , wherein the one or more instance specific parameters of the second pipeline template include at least one of a service name, an administrative contact, a run-as username for one of the plurality of computing services included in the deployment pipeline.
3. The computer-readable storage medium of claim 1 , wherein the pipeline synthesis drivers include at least one of a pipeline driver, a deployment driver, a host driver, a network driver, a security driver, an identity and access management (IAM) driver, a performance monitor driver, and an alarm driver.
4. The computer-readable storage medium of claim 1 , wherein the application definition is formatted according to a structured interchange format.
5. The computer-readable storage medium of claim 1 , wherein the first base pipeline template extends at least a third pipeline template.
6. A system, comprising:
a processor; and
a memory storing one or more applications, which when executed on the processor, perform an operation to generate an application definition modeling a deployment pipeline used to deploy changes to a production computing service, the operation comprising:
identifying an instance of a live pipeline template (LPT), wherein the instance of the LPT specializes one or more base pipeline templates with instance specific parameters for a plurality of deployment stages of the deployment pipeline,
building an application target from the instance of the LPT,
invoking the application target to generate the application definition, wherein the application definition provides a fully-specified configuration for each of a plurality of computing services included in the deployment pipeline; and
launching respective first and second instances of the deployment pipeline in first and second cloud computing regions by invoking, for each computing service referenced in the application definition, a respective pipeline synthesis driver to configure one of the plurality of computing services in the first and second cloud computing regions according to the fully-specified configuration provided in the application definition.
7. The system of claim 6 , wherein the pipeline synthesis drivers are invoked in an order determined to satisfy one or more service dependencies among the pipeline synthesis drivers.
8. The system of claim 6 , wherein the first cloud computing region is selected based at least in part on the instance specific parameters used to specialize the one or more base pipeline templates.
9. The system of claim 6 , wherein at least a first one of the base pipeline templates specifies configuration parameters for each of the plurality of deployment stages of the deployment pipeline.
10. The system of claim 6 , wherein a second one of the base pipeline templates extends at least a first one of the one or more base pipeline templates and wherein the second base pipeline template specifies requirements for the deployment pipeline which are associated with a production service type.
11. The system of claim 6 , wherein the instance specific parameters include at least one of a service name, an administrative contact, a run-as username for one of the plurality of computing services included in the deployment pipeline.
12. The system of claim 6 , wherein the pipeline synthesis drivers include at least one of a pipeline driver, a deployment driver, a host driver, a network driver, a security driver, an identity access and management (IAM) driver, a performance monitor driver, and an alarm driver.
13. The system of claim 6 , wherein the application definition is formatted according to a structured interchange format.
14. A computer-implemented method for provisioning a deployment pipeline, the method comprising:
determining that a plurality of computing services referenced by an application definition modeling the deployment pipeline are available to host a first instance of the deployment pipeline, wherein the application definition is generated from an instance of a live pipeline template (LPT) specializing a base pipeline template with instance specific parameters for a plurality of deployment stages of the deployment pipeline, and wherein the application definition provides a fully-specified configuration for each of the plurality of computing services referenced in the application definition; and
launching the first instance of the deployment pipeline in a first cloud computing region and a second instance of the deployment pipeline in a second cloud computing region by invoking, for each computing service referenced in the application definition, a respective pipeline synthesis driver to configure one of the plurality of computing services according to the fully-specified configuration provided in the application definition.
15. The method of claim 14 , wherein the instance specific parameters include at least one of a service name, an administrative contact, a run-as username for one of the plurality of computing services included in the deployment pipeline.
16. The method of claim 14 , wherein the pipeline synthesis drivers include at least one of a pipeline driver, a deployment driver, a host driver, a network driver, a security driver, an identity and access management (IAM) driver, a performance monitor driver, and an alarm driver.
17. The method of claim 14 , wherein the instance of the deployment pipeline is a continuous deployment pipeline used to deploy changes to a production computing service.
18. The method of claim 14 , wherein the application definition is formatted according to a structured interchange format.