Scaling capacity of data warehouses to user-defined levels
A system and method of scaling capacity of data warehouses to user-defined levels. The method includes provisioning a data warehouse including a plurality of processing resources, the plurality of processing resources including at least one processor and at least one storage device. The method includes receiving a request to process database data stored on a storage platform including a plurality of shared storage devices in association with the data warehouse, wherein the request indicates a performance level for processing the request. The method includes determining that a capacity of the plurality of processing resources of the data warehouse would reach a threshold capacity when processing the request according to the performance level. The method includes increasing the capacity of the data warehouse for the data warehouse to process the request according to the performance level.
1. A method comprising:
provisioning a data warehouse comprising a plurality of processing resources, the plurality of processing resources comprising at least one processor and at least one storage device;
receiving a request to process database data stored on a storage platform comprising a plurality of shared storage devices in association with the data warehouse, wherein the request indicates a performance level for processing the request and providing a query response within a particular time period;
receiving, from a user device, a statement indicating a performance of the processed request as observed by the user device;
determining, based on the statement, that a capacity of the plurality of processing resources of the data warehouse reached a threshold capacity when processing the request according to the performance level; and
increasing the capacity of the data warehouse for the data warehouse to process the request according to the performance level.
2. The method of claim 1 , wherein increasing the capacity of the data warehouse for the data warehouse to process the request according to the performance level comprises at least one of:
adding at least one additional processor to the data warehouse; or
adding at least one additional storage device to the data warehouse.
3. The method of claim 2 , further comprising:
determining that the at least one additional processor is no longer needed; and
removing the at least one additional processor from the data warehouse.
4. The method of claim 2 , wherein adding the at least one additional processor to the data warehouse is accomplished without affecting a storage capacity of the plurality of shared storage devices.
5. The method of claim 1 , wherein determining that the capacity would reach a threshold capacity comprises determining that a current query processing delay would exceed a threshold delay.
6. The method of claim 1 , wherein each processing resource comprises a plurality of processors and a plurality of storage devices.
7. The method of claim 1 , wherein the storage platform is physically separate from the plurality of processing resources of the data warehouse.
8. The method of claim 1 , wherein the request comprises a data retrieval request or a data storage request.
9. A system, comprising:
a processing device having a hardware processor executing instructions programmed to:
provision a data warehouse comprising a plurality of processing resources, the plurality of processing resources comprising at least one processor and at least one storage device;
receive a request to process database data stored on a storage platform comprising a plurality of shared storage devices in association with the data warehouse, wherein the request indicates a performance level for processing the request and providing a query response within a particular time period;
receive, from a user device, a statement indicating a performance of the processed request as observed by the user device;
determine, based on the statement, that a capacity of the plurality of processing resources of the data warehouse a threshold capacity when processing the request according to the performance level; and
increase the capacity of the data warehouse for the data warehouse to process the request according to the performance level.
10. The system of claim 9 , wherein the processing device is further programmed to:
add at least one additional processor to the data warehouse; or
add at least one additional storage device to the data warehouse.
11. The system of claim 10 , wherein the processing device is further programmed to:
determine that the at least one additional processor is no longer needed; and
remove the at least one additional processor from the data warehouse.
12. The system of claim 10 , wherein to add the at least one additional processor to the data warehouse is accomplished without affecting a storage capacity of the plurality of shared storage devices.
13. The system of claim 9 , wherein to determine that the capacity would reach a threshold capacity comprises to determine that a current query processing delay would exceed a threshold delay.
14. The system of claim 9 , wherein each processing resource comprises a plurality of processors and a plurality of storage devices.
15. The system of claim 9 , wherein the storage platform is physically separate from the plurality of processing resources of the data warehouse.
16. The system of claim 9 , wherein the request comprises a data retrieval request or a data storage request.
17. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to:
provision a data warehouse comprising a plurality of processing resources, the plurality of processing resources comprising at least one processor and at least one storage device;
receive a request to process database data stored on a storage platform comprising a plurality of shared storage devices in association with the data warehouse, wherein the request indicates a performance level for processing the request and providing a query response within a particular time period;
receive, from a user device, a statement indicating a performance of the processed request as observed by the user device;
determine, based on the statement, that a capacity of the plurality of processing resources of the data warehouse reached a threshold capacity when processing the request according to the performance level; and
increase the capacity of the data warehouse for the data warehouse to process the request according to the performance level.
18. The non-transitory computer-readable medium of claim 17 , wherein the instructions further cause the processor to:
add at least one additional processor to the data warehouse; or
add at least one additional storage device to the data warehouse.
19. The non-transitory computer-readable medium of claim 18 , wherein the instructions further cause the processor to:
determine that the at least one additional processor is no longer needed; and
remove the at least one additional processor from the data warehouse.
20. The non-transitory computer-readable medium of claim 18 , wherein adding the at least one additional processor to the data warehouse is accomplished without affecting a storage capacity of the plurality of shared storage devices.
21. The non-transitory computer-readable medium of claim 17 , wherein determining that the capacity would reach a threshold capacity comprises determining that a current query processing delay would exceed a threshold delay.
22. The non-transitory computer-readable medium of claim 17 , wherein each processing resource comprises a plurality of processors and a plurality of storage devices.
23. The non-transitory computer-readable medium of claim 17 , wherein the storage platform is physically separate from the plurality of processing resources of the data warehouse.
24. The non-transitory computer-readable medium of claim 17 , wherein the request comprises a data retrieval request or a data storage request.