Resource provisioning systems and methods
Example resource provisioning systems and methods are described. In one implementation, multiple processing resources are provided within a data warehouse. The processing resources include at least one processor and at least one storage device. At least one query to process database data is received. At least some of the processing resources may process the database data. When a processing capacity of the processing resources has reached a threshold processing capacity, the processing capacity is automatically scaled by adding at least one additional processor to the data warehouse.
1. A method comprising:
providing a plurality of computation resources, the plurality of computation resources comprising at least one processing core and at least one storage device;
receiving one or more processing requests to process database data stored on a storage platform comprising a plurality of share storage devices in association with the plurality of computation resources, wherein at least some of the plurality of computation resources process the database data;
providing metadata, wherein the metadata allows the plurality of computation resources to determine whether a piece of data should be accessed from the plurality of share storage devices;
periodically reporting statistics regarding the processing capacity of the plurality of computation resources and the storage capacity associated with the plurality of computation resources; and
automatically scaling the processing capacity of the plurality of computation resources and the storage capacity of the storage platform based on the reported statistics simultaneously by adding additional computation resources to the cluster and by adding at least one additional storage device.
2. The method of claim 1 , wherein the plurality of resources are provided within a cluster.
3. The method of claim 1 , wherein the reporting statistics further comprises determining whether the processing capacity of the plurality of computation resources has reached a threshold processing capacity, and the scaling the processing capacity further comprises scaling based upon whether the threshold has been reached.
4. The method of claim 1 , wherein adding additional computation resources comprises adding at least one additional processing core to process the one or more processing requests to process the database data stored in association with the plurality of computation resources.
5. The method of claim 4 , further comprising:
determining that the at least one additional processing core is no longer needed; and
removing the at least one additional processing core from the plurality of computing resources.
6. The method of claim 4 , wherein the storage platform is independent of the plurality of computation resources.
7. The method of claim 6 , wherein adding the at least one additional processing core to the cluster is accomplished without affecting a storage capacity of the plurality of shared storage devices.
8. The method of claim 1 , wherein automatically scaling the processing capacity of the plurality of computation resources further comprises scaling a storage capacity of the plurality of computation resources by adding at least one additional storage device.
9. The method of claim 1 , wherein determining that the processing capacity has reached a threshold processing capacity comprises determining that a current job processing delay has exceeded a threshold delay.
10. The method of claim 1 , wherein each of the plurality of computation resources comprises a processing core and a storage device.
11. A system, comprising:
a memory; and
a processor operatively coupled to the memory, the processor to:
provide a plurality of computation resources, the plurality of computation resources comprising at least one processing core and at least one storage device;
receive one or more processing requests to process database data stored on a storage platform comprising a plurality of share storage devices in association with the plurality of computation resources, wherein at least some of the plurality of computation resources process the database data;
provide metadata, wherein the metadata allows the plurality of computation resources to determine whether a piece of data should be accessed from the plurality of share storage devices;
periodically report statistics regarding the processing capacity of the plurality of computation resources and the storage capacity associated with the plurality of computation resources; and
automatically scale the processing capacity of the plurality of computation resources and the storage capacity of the storage platform based on the reported statistics simultaneously by adding additional computation resources to the cluster and by adding at least one additional storage device.
12. The system of claim 11 , wherein the plurality of resources are provided within a cluster.
13. The system of claim 11 , wherein to report the statistics the processor is further to determine whether the processing capacity of the plurality of computation resources has reached a threshold processing capacity, and to scale the processing capacity the processor is further to scale based upon whether the threshold has been reached.
14. The system of claim 11 , wherein to add additional computation resources the processor is to add at least one additional processing core to process the one or more processing requests to process the database data stored in association with the plurality of computation resources.
15. The system of claim 14 , wherein the processor is further to:
determine that the at least one additional processing core is no longer needed; and
remove the at least one additional processing core from the plurality of computing resources.
16. The system of claim 14 , wherein the storage platform is independent of the plurality of computation resources.
17. The system of claim 16 , wherein the processor adds the at least one additional processing core to the cluster without affecting a storage capacity of the plurality of shared storage devices.
18. The system of claim 11 , wherein to automatically scale the processing capacity of the plurality of computation resources the processor is to scale a storage capacity of the plurality of computation resources by adding at least one additional storage device.
19. The system of claim 11 , wherein to determine that the processing capacity has reached a threshold processing capacity the processor is to determine that a current job processing delay has exceeded a threshold delay.
20. The system of claim 11 , wherein each of the plurality of computation resources comprises a processing core and a storage device.
21. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to:
provide a plurality of computation resources, the plurality of computation resources comprising at least one processing core and at least one storage device;
receive one or more processing requests to process database data stored on a storage platform comprising a plurality of share storage devices in association with the plurality of computation resources, wherein at least some of the plurality of computation resources process the database data;
provide metadata, wherein the metadata allows the plurality of computation resources to determine whether a piece of data should be accessed from the plurality of share storage devices;
periodically report statistics regarding the processing capacity of the plurality of computation resources and the storage capacity associated with the plurality of computation resources; and
automatically scaling the processing capacity of the plurality of computation resources and the storage capacity of the storage platform based on the reported statistics simultaneously by adding additional computation resources to the cluster and by adding at least one additional storage device.
22. The non-transitory computer-readable medium of claim 21 , wherein the plurality of resources are provided within a cluster.
23. The non-transitory computer-readable medium of claim 21 , wherein to report the statistics the processor is further to determine whether the processing capacity of the plurality of computation resources has reached a threshold processing capacity, and to scale the processing capacity the processor is further to scale based upon whether the threshold has been reached.
24. The non-transitory computer-readable medium of claim 21 , wherein to add additional computation resources the processor is to add at least one additional processing core to process the one or more processing requests to process the database data stored in association with the plurality of computation resources.
25. The non-transitory computer-readable medium of claim 24 , wherein the processor is further to:
determine that the at least one additional processing core is no longer needed; and
remove the at least one additional processing core from the plurality of computing resources.
26. The non-transitory computer-readable medium of claim 24 , wherein the storage platform is independent of the plurality of computation resources.
27. The non-transitory computer-readable medium of claim 26 , wherein the processor adds the at least one additional processing core to the cluster without affecting a storage capacity of the plurality of shared storage devices.
28. The non-transitory computer-readable medium of claim 21 , wherein to automatically scale the processing capacity of the plurality of computation resources the processor is to scale a storage capacity of the plurality of computation resources by adding at least one additional storage device.
29. The non-transitory computer-readable medium of claim 21 , wherein to determine that the processing capacity has reached a threshold processing capacity the processor is to determine that a current job processing delay has exceeded a threshold delay.
30. The non-transitory computer-readable medium of claim 21 , wherein each of the plurality of computation resources comprises a processing core and a storage device.