On-demand provisioning of customized developer environments
Systems and techniques are provided for managing and creating customized testing and development environments by a custom environment manager for computer or data management systems. In a specific implementation, the custom environment manager includes request receivers that receive orders associated with a prioritization from custom environment requesters and store the received orders in a request queue that holds multiple orders having different prioritizations and made by different requestors. The custom environment manager also includes pooled resource managers that track available virtual and physical computing resources needed to build custom environments. The custom environment manager also includes configurators that create custom environments based upon prioritization of the orders and available resources and return the custom environment to the requestor of the order.
1. A system for managing custom environments, including:
one or more processors coupled with memory storing instructions that, when executed, perform actions implementing:
one or more request receivers that receive orders from multiple requesters, including a first order from a first requestor requesting a custom environment or modification to a custom environment and a specified test scenario to be performed, and store the first order in a request queue that holds multiple orders having different prioritizations and made by the multiple requesters;
one or more pooled resource managers that track available resources and unavailable resources in a resource pool comprising at least one or more virtual resources and at least one or more physical resources deployed on machines external to the system; and
one or more configurators that create custom environments based upon multiple orders received by:
retrieving an order in the request queue and allocating available virtual resources and physical resources from the resource pool to create or customize the custom environment specified in the order, the allocating including allocating a first virtual resource of the at least one or more virtual resources and a first physical resource of the at least one or more physical resources to meet requirements of the specified test scenario, and commence returning the custom environment to the first requestor of the first order; and
wherein the one or more pooled resource managers is configured to automatically recover the first virtual resource and the first physical resource in the custom environment when tasks of the specified test scenario using the first physical resource and the first virtual resource ordered are complete, the recovering including closing one or more nodes associated with the first virtual resource allocated and freeing the first physical resource allocated.
2. The system of claim 1 , further including a recipe database and further including retrieving the first virtual resource and a first physical resource configuration corresponding to the first physical resource stored in the recipe database; and
allocating the first virtual resource and the first physical resource according to the first virtual resource and the first physical resource configuration retrieved from the recipe database.
3. The system of claim 1 , wherein allocating the first virtual resource and the first physical resource includes creating one or more nodes of a virtual type in the first virtual resource and a physical type in the first physical resource and populating the one or more nodes with data sources.
4. The system of claim 1 , wherein allocating the first virtual resource and the first physical resource includes locating one or more nodes of a virtual type available in the first virtual resource in the resource pool and one or more nodes of a physical type available in the first physical resource in the resource pool and populating the one or more nodes of the virtual type and the one or more nodes of the physical type with data sources.
5. The system of claim 1 , wherein the instructions further include allocating resources to subsequent orders from a second requestor at a lower prioritization than the first order by a first requester when quota for the second requestor is depleted.
6. The system of claim 1 , wherein the instructions further include:
receiving from a second requestor a second order, wherein a prioritization of the second order is greater than a prioritization of the first order;
suspending processing of the first order; and
allocating resources to the second order from the second requestor at a higher priority level than the first order until occurrence of either quota for the second requestor is depleted or the second order is completed.
7. The system of claim 1 , wherein the instructions further include receiving an update communicating one or more changes to an order, including at least one of adding comments, extending a time limit of the order, changing requesters of the order, closing the order, and adding more physical resources of the at least one or more physical resources and more virtual resources of the at least one or more virtual resources to the order.
8. The system of claim 1 , wherein the instructions further include creating a pre-bootstrapped custom environment image for immediate deployment and saving the pre-bootstrapped custom environment image for subsequent provisioning.
9. The system of claim 1 , wherein the instructions further include retrieving a saved pre-bootstrapped custom environment image and creating one or more custom environments.
10. The system of claim 1 , further including a centralized collector to collect and store data across multiple custom environments, and present data collected by creating one or more dashboard displays to present data visualize collected in a centralized location.
11. The system of claim 1 , wherein the instructions further include predicting future orders, anticipating requirements for the future orders predicted and preparing the one or more nodes in virtual resources and physical resources to be allocated to meet the requirements anticipated.
12. The system of claim 1 , wherein the instructions further include testing functionality of a resource prior to allocating the resource to an order, and when a resource is determined to be inoperable, determining a replacement resource to be allocated to the order.
13. A method for managing custom environments, including:
at a computer system, receiving orders from multiple requestors, including a first order from a first requester requesting a custom environment or modification to a custom environment and a specified test scenario to be performed, and storing the first order in a request queue that holds multiple orders having different prioritizations and made by the multiple requestors;
at the computer system, tracking available resources and unavailable resources in a resource pool comprising of at least one or more virtual resources and at least one or more physical resources deployed on machines external to the computer system; and
at the computer system, creating custom environments based upon orders received by:
retrieving an order in the request queue and allocating available virtual resources and physical resources from the resource pool to create or customize the custom environment specified in the order, the allocating including allocating a first virtual resource of the at least one or more virtual resources and a first physical resource of the at least one or more physical resources to meet requirements of the specified test scenario; and commence returning the custom environment to the first requestor of the first order; and
at the computer system, automatically recovering the first virtual resource and the first physical resource in the custom environment when tasks of the specified test scenario using the first physical resource and the first virtual resource ordered are complete, the recovering includes closing one or more nodes associated with the first virtual resource allocated and freeing the first physical resource allocated.
14. The method of claim 13 , further including providing a recipe database and further including retrieving a first virtual resource and a first physical resource configuration corresponding to the first physical resource stored in the recipe database; and
allocating the first virtual resource and the first physical resource according to the first virtual resource and the first physical resource configurations retrieved from the recipe database.
15. The method of claim 13 , wherein allocating the first virtual resource and the first physical resource includes creating one or more nodes of a virtual type in the first virtual resource and a physical type in the first physical resource and populating the one or more nodes with data sources.
16. The method of claim 13 , wherein allocating the first virtual resource and first physical resource includes locating one or more nodes of a virtual type available in the first virtual resource in the resource pool and one or more nodes of a physical type available in the first physical resource in the resource pool and populating the one or more nodes of the virtual type and the one or more nodes of the physical type with data sources.
17. The method of claim 13 , further including allocating resources to subsequent orders from a second requestor at a lower prioritization than the first order by a first requester when quota for the second requestor is depleted.
18. The method of claim 13 , further including
receiving from a second requestor a second order, wherein a prioritization of the second order is greater than a prioritization of the first order;
suspending processing of the first order; and
allocating resources to the second order from the second requestor at a higher priority level than the first order until occurrence of either quota for the second requestor is depleted or the second order is completed.
19. The method of claim 13 , further including: receiving an update communicating one or more changes to an order, including at least one of adding comments, extending a time limit of the order, changing requesters of the order, closing the order, and adding more physical resources of the at least one or more physical resources and more virtual resources of the at least one or more virtual resources to the order.
20. The method of claim 13 , further including creating a pre-bootstrapped custom environment image for immediate deployment and saving the pre-bootstrapped custom environment image for subsequent provisioning.
21. The method of claim 20 , further including retrieving a saved pre-bootstrapped custom environment image and creating one or more custom environments.
22. The method of claim 13 , further including providing a centralized collector to collect and store data across multiple custom environments, and presenting data collected by creating one or more dashboard displays to present data visualize collected in a centralized location.
23. The method of claim 13 , further including predicting future orders, anticipating requirements for the future orders predicted and preparing the one or more nodes in virtual resource and physical resource to be allocated to meet the requirements anticipated.
24. The method of claim 13 , further including testing functionality of a resource prior to allocating the resource to an order, and when a resource is determined to be inoperable, determining a replacement resource to be allocated to the order.
25. A non-transitory machine readable medium storing instructions which, when executed by one or more processors, cause the machine to implement operations for creating and managing custom environments, the operations including:
receiving orders from multiple requesters, including a first order from a first requester requesting a custom environment or modification to a custom environment and a specified test scenario to be performed, and storing the first order in a request queue that holds multiple orders having different prioritizations and made by the multiple requestors;
tracking available resources and unavailable resources in a resource pool comprising of at least one or more virtual resources and at least one or more physical resources deployed on machines external to a computer system; and
creating custom environments based upon orders received by:
retrieving an order in the request queue and allocating available virtual resources and physical resources from the resource pool to create or customize the custom environment specified in the order, the allocating, including allocating a first virtual resource of the at least one or more virtual resources and a first physical resource of the at least one or more physical resources to meet requirements of the specified test scenario, and commence returning the custom environment to the first requestor of the first order; and
automatically recovering the first virtual resource and the first physical resource in the custom environment when tasks of the specified test scenario using the first physical resource and the first virtual resource ordered are complete, the recovering including closing one or more nodes associated with the first virtual resource allocated and freeing the first physical resource allocated.