Intelligent analytic cloud provisioning
A services platform acts as an intermediary between an existing enterprise analytic environment, and one or more underlying cloud service providers. The platform provides enterprise “big data-as-a-service,” by which an enterprise can seamlessly and easily provision new capacity for processing its analytic workload, and migrate data sources (e.g., data warehouse marts, enterprise data warehouses, analytic sandboxes, and the like) to the cloud for processing. The platform provides end-to-end enterprise class manageability of enterprise data assets, from data collection, aggregation, movement, staging and processing, all while providing service levels, security, access and governance. The platform integrates directly but seamlessly into the enterprise analytic stack, and existing analytics applications work as normal. The platform provides a way for the enterprise to translate its workloads into clusters of compute resources that meet its service level requirements.
1. Apparatus associated with a cloud computing infrastructure, comprising:
one or more hardware processors;
computer memory holding computer program instructions executing in the one or more hardware processors, the computer program instructions operative to identify resources in the cloud computing infrastructure on which an analytics workload is to be executed by:
generating a set of workload resource requirements necessary to support a workload at one of: a desired service level, and a desired cost;
mapping the set of workload resource requirements onto a resource vector space, the workload resource requirements being represented in the resource vector space as graphical objects;
performing a cluster analysis on the graphical objects to identify, based on graphical objects that cluster with one another in the resource vector space as a group, one or more analytics workloads requiring similar resources in the cloud computing infrastructure; and
matching the set of workload resource requirements in a cluster against a set of resource bundling options available in the cloud computing infrastructure, wherein a resource bundling option comprises a cloud vendor, and a description of types and amounts of resources provided in the resource bundling option; and
outputting descriptions of one or more resource bundling options that, based on the matching, will support the analytics workload and the desired service level.
2. The apparatus as described in claim 1 wherein the workload resource requirements are generated from historical workload information.
3. The apparatus as described in claim 1 wherein the workload resource requirements are generated deterministically.
4. The apparatus as described in claim 1 wherein the resource bundling options are identified via an application programming interface (API).
5. The apparatus as described in claim 1 wherein the descriptions identify an ordered list of resource bundling options.
6. The apparatus as described in claim 5 wherein the ordered list includes a first entry that is a lowest cost option.
7. The apparatus as described in claim 6 wherein at least one entry in the ordered list also includes a probability of missing the service level.
8. Apparatus associated with a cloud computing infrastructure, comprising:
one or more hardware processors;
computer memory holding computer program instructions executing in the one or more hardware processors, the computer program instructions operative to identify resources in the cloud computing infrastructure on which an analytics workload is to be executed by:
generating a set of workload resource requirements necessary to support a workload;
mapping the set of workload resource requirements onto a resource vector space, the workload resource requirements being represented in the resource vector space as graphical objects;
performing a cluster analysis on the graphical objects to identify, based on graphical objects that cluster with one another in the resource vector space as a group, one or more analytics workloads requiring similar resources in the cloud computing infrastructure;
providing information specifying an initial cluster configuration of at least one cluster as determined by the cluster analysis;
as an analytics workload is executing in the initial cluster configuration, monitoring performance of the initial cluster configuration against a service level; and
based on the monitoring, providing supplemental information specifying a modified cluster configuration to thereby adjust the resources in the cloud computing infrastructure.
9. The apparatus as described in claim 8 wherein the monitoring is carried out continuously as the analytics workload is executing.
10. The apparatus as described in claim 8 wherein the workload resource requirements are generated from historical workload information.
11. The apparatus as described in claim 8 wherein the workload resource requirements are generated deterministically.
12. The apparatus as described in claim 1 wherein the set of resource bundling options describe available resource bundles, each resource bundle providing different amounts of different resources.
13. The apparatus as described in claim 12 wherein each resource bundling option is represented as a vector in the resource vector space.
14. The apparatus as described in claim 13 wherein the resource vector space is a multi-dimensional resource vector space.
15. The apparatus as described in claim 1 wherein the graphical objects are circles.