Automated construction of software pipeline
A controller generates a software pipeline in response to obtaining an instruction to build the software pipeline for a first user. The controller divides the software pipeline into a plurality of tasks and determining a set of resources for performing each task in the plurality of tasks. The controller reserves the set of resources based on a policy associated with the first user. The controller causes the first software pipeline to run using the set of resources reserved for performing each task in the plurality of tasks.
1. A computer-implemented method comprising:
obtaining an instruction to build a first software pipeline for a first user associated with a first tenant;
dividing the first software pipeline into a plurality of tasks, wherein the plurality of tasks includes a plurality of request tasks;
retrieving policy information for the first tenant;
determining a set of resources for performing each task in the plurality of tasks based on a service level offering defined in the policy information for the first tenant;
associating each of the plurality of tasks with a corresponding tag, wherein each of the plurality of request tasks is associated with a corresponding Type-Length-Value (TLV) tag having a plurality of bits representing Enhanced Transmission Selection (ETS) parameters or Priority Flow Control (PFC) parameters;
placing the plurality of request tasks into one or more delivery queues;
shuffling the one or more delivery queues based on a priority level associated with each of the plurality of request tasks and a policy associated with the first user;
allocating the set of resources for performing each task based on the policy associated with the first user;
running the first software pipeline using the set of resources allocated for performing each task in the plurality of tasks;
automatically tracking a performance of each task in the plurality of tasks with the corresponding tag; and
analyzing previous execution history and the corresponding tag for each task in the plurality of tasks to determine whether to assign a different tag to each task.
2. The method of claim 1 , wherein at least one task in the plurality of tasks is shared across a plurality of software pipelines.
3. The method of claim 1 , wherein each corresponding tag identifies a corresponding task across a plurality of software pipelines.
4. The method of claim 1 , wherein automatically tracking the performance of each task in the plurality of tasks further comprises:
processing the performance of each task in the plurality of tasks across a plurality of software pipelines using machine learning.
5. The method of claim 1 , further comprising adjusting the set of resources allocated for performing one or more tasks in the plurality of tasks based on a relative priority of the first software pipeline to a second software pipeline, wherein the second software pipeline is built for a second user associated with a second tenant different than the first tenant.
6. The method of claim 1 , wherein the set of resources for performing each task comprises network resources or processing resources.
7. The method of claim 1 , wherein the first software pipeline includes a Continuous Integration (CI) operation or a Continuous Deployment (CD) operation.
8. The method of claim 1 , wherein the TLV tag further includes one or more bits representing one or more of:
a plurality of Quality of Service (QOS) levels,
a plurality of processing levels,
one or more functions of a Continuous Integration (CI) operation,
one or more functions of a Continuous Deployment (CD) operation, or
a thread priority parameter identifying one or more central processing unit (CPU) cores and one or more CPU cycles allocated to each task in the plurality of tasks.
9. An apparatus comprising:
a network interface configured to communicate with a plurality of computing devices; and
a processor coupled to the network interface, the processor configured to:
obtain an instruction via the network interface, the instruction indicating a first software pipeline to build for a first user associated with a first tenant;
divide the first software pipeline into a plurality of tasks, wherein the plurality of tasks includes a plurality of request tasks;
retrieve policy information for the first tenant via the network interface;
determine a set of resources for performing each task in the plurality of tasks based on a service level offering defined in the policy information for the first tenant;
associate each of the plurality of tasks with a corresponding tag, wherein each of the plurality of request tasks is associated with a corresponding Type-Length-Value TLV) tag having a plurality of bits representing Enhanced Transmission Selection (ETS) parameters or Priority Flow Control (PFC) parameters;
place the plurality of request tasks into one or more delivery queues;
shuffle the one or more delivery queues based on a priority level associated with each of the plurality of request tasks and a policy associated with the first user;
allocate the set of resources for performing each task based on the policy associated with the first user;
cause the first software pipeline to run using the set of resources allocated for performing each task in the plurality of tasks;
automatically track a performance of each task in the plurality of tasks with the corresponding tag; and
analyze previous execution history and the corresponding tag for each task in the plurality of tasks to determine whether to assign a different tag to each task.
10. The apparatus of claim 9 , wherein at least one task in the plurality of tasks is shared across a plurality of software pipelines.
11. The apparatus of claim 9 , wherein each corresponding tag identifies a corresponding task across a plurality of software pipelines.
12. The apparatus of claim 9 , wherein the processor is configured to automatically track the performance of each task in the plurality of tasks by:
processing the performance of each task in the plurality of tasks across a plurality of software pipelines using machine learning.
13. The apparatus of claim 9 , wherein the processor is further configured to adjust the set of resources allocated for performing one or more tasks in the plurality of tasks based on a relative priority of the first software pipeline to a second software pipeline, wherein the processor is configured to build the second software pipeline for a second user associated with a second tenant different than the first tenant.
14. The apparatus of claim 9 , wherein the processor is configured to allocate the set of resources for performing each task by allocating network resources or processing resources.
15. The apparatus of claim 9 , wherein the first software pipeline includes a Continuous Integration (CI) operation or a Continuous Deployment (CD) operation.
16. The apparatus of claim 9 , wherein the TLV tag further includes one or more bits representing one or more of:
a plurality of Quality of Service (QOS) levels,
a plurality of processing levels,
one or more functions of a Continuous Integration (CI) operation,
one or more functions of a Continuous Deployment (CD) operation, or
a thread priority parameter identifying one or more central processing unit (CPU) cores and one or more CPU cycles allocated to each task in the plurality of tasks.
17. One or more non-transitory computer readable storage media encoded with software comprising computer executable instructions and, when the software is executed, it is operable to cause a processor to:
obtain an instruction to build a first software pipeline for a first user associated with a first tenant;
divide the first software pipeline into a plurality of tasks, wherein the plurality of tasks includes a plurality of request tasks;
retrieve policy information for the first tenant;
determine a set of resources for performing each task in the plurality of tasks based on a service level offering defined in the policy information for the first tenant;
associate each of the plurality of tasks with a corresponding tag, wherein each of the plurality of request tasks is associated with a corresponding Type-Length-Value (TLV) tag having a plurality of bits representing Enhanced Transmission Selection (ETS) parameters or Priority Flow Control (PFC) parameters;
place the plurality of request tasks into one or more delivery queues;
shuffle the one or more delivery queues based on a priority level associated with each of the plurality of request tasks and a policy associated with the first user;
allocate the set of resources for performing each task based on the policy associated with the first user;
run the first software pipeline using the set of resources allocated for performing each task in the plurality of tasks;
automatically track a performance of each task in the plurality of tasks with a corresponding tag; and
analyze previous execution history and the corresponding tag for each task in the plurality of tasks to determine whether to assign a different tag to each task.
18. The one or more non-transitory computer readable storage media of claim 17 , wherein at least one task in the plurality of tasks is shared across a plurality of software pipelines.
19. The one or more non-transitory computer readable storage media of claim 17 , wherein each corresponding tag identifies a corresponding task across a plurality of software pipelines.
20. The one or more non-transitory computer readable storage media of claim 17 , wherein the software is further operable to cause the processor to automatically track the performance of each task in the plurality of tasks by:
processing the performance of each task in the plurality of tasks across a plurality of software pipelines using machine learning.
21. The one or more non-transitory computer readable storage media of claim 17 , wherein the software is further operable to cause the processor to adjust the set of resources allocated for performing one or more tasks in the plurality of tasks based on a relative priority of the first software pipeline to a second software pipeline, wherein the second software pipeline is built for a second user associated with a second tenant different than the first tenant.
22. The one or more non-transitory computer readable storage media of claim 17 , wherein the software is further operable to cause the processor to allocate the set of resources for performing each task by allocating network resources or processing resources.
23. The one or more non-transitory computer readable storage media of claim 17 , wherein the first software pipeline includes a Continuous Integration (CI) operation or a Continuous Deployment (CD) operation.