Resource scaling for distributed database services
Described embodiments provide systems and methods for resource scaling. A computing device may include a data processor. The data processor may receive, from a distributed database service (DDS), usage data on resource request-units from a plurality of services that use resources provided by the DDS. The computing device may include a transform agent. The transform agent may transform at least a subset of the N sets of data, into at least three transformed datasets each corresponding to a different frequency range. The computing device may include a predictor. The predictor may provide a predicted value of request-units according to the at least three transformed datasets. The predictor may request the DDS to provide the resources up to a number of request-units at a given time instance according to the predicted value.
1. A computing device, comprising:
a data processor configured to:
receive, from a distributed database service (DDS), usage data on resource request-units from a plurality of services that use resources provided by the DDS; and
provide a plurality of datasets corresponding to a plurality of time windows, using the usage data;
a transform agent configured to transform at least a subset of the plurality of datasets, into at least three transformed datasets each corresponding to a different frequency range; and
a predictor configured to:
provide a predicted value of request-units according to the at least three transformed datasets; and
request the DDS to provide the resources up to a number of request-units at a given time instance according to the predicted value.
2. The computing device of claim 1 , wherein the data processor is configured to store the received usage data into a queue in memory.
3. The computing device of claim 1 , wherein the data processor is further configured to perform a map-and-reduce of at least a portion of the usage data in real time, to provide a first set of the plurality of datasets.
4. The computing device of claim 1 , wherein the transform agent is configured to transform the at least a subset of the plurality of datasets using Wavelet Transform.
5. The computing device of claim 1 , wherein the transform agent is configured to process the at least a subset of the plurality of datasets as a stationary stochastic process.
6. The computing device of claim 1 , wherein the at least three transformed datasets correspond to a dataset indicative of a trend, a dataset indicative of cyclic data, and a dataset indicative of noise.
7. The computing device of claim 1 , wherein the predictor is configured to provide the predicted value using at least one of: autoregressive-moving average model (ARMA) method, Kalman filtering, or Holt's Linear Trend method.
8. The computing device of claim 1 , wherein the predictor is configured to perform a prediction on each of the at least three transformed datasets, and to provide the predicted value of request-units according to the predictions.
9. The computing device of claim 1 , wherein the predictor is configured to:
determine a standard deviation of the at least a subset of the plurality of datasets; and
request the DDS to provide the resources up to an updated number of request-units at a given time instance according to the predicted value bounded by twice the standard deviation.
10. A method, comprising:
receiving, from a distributed database service (DDS), usage data on resource request-units from a plurality of services that use resources provided by the DDS;
providing a plurality of datasets corresponding to a plurality of time windows, using the usage data;
transforming at least a subset of the plurality of datasets, into at least three transformed datasets each corresponding to a different frequency range;
providing a predicted value of request-units according to the at least three transformed datasets; and
requesting the DDS to provide the resources up to a number of request-units at a given time instance according to the predicted value.
11. The method of claim 10 , comprising storing the received usage data into a queue in memory.
12. The method of claim 10 , further comprising performing a map-and-reduce of at least a portion of the usage data in real time, to provide a first set of the plurality of datasets.
13. The method of claim 10 , comprising transforming the at least the subset of the plurality of datasets using Wavelet Transform.
14. The method of claim 10 , comprising processing the at least the subset of the plurality of datasets as a stationary stochastic process.
15. The method of claim 10 , wherein the at least three transformed datasets correspond to a dataset indicative of a trend, a dataset indicative of cyclic data, and a dataset indicative of noise.
16. The method of claim 10 , comprising providing the predicted value using autoregressive-moving average model (ARMA) method, Kalman filtering, or Holt's Linear Trend method.
17. The method of claim 10 , further comprising performing a prediction on each of the at least three transformed datasets, and providing the predicted value of request-units according to the predictions.
18. The method of claim 10 , further comprising:
determining a standard deviation of the at least a subset of the plurality of datasets of processed usage data; and
requesting the DDS to provide the resources up to an updated number of request-units at a given time instance according to the predicted value bounded by twice the standard deviation.
19. A non-transitory computer readable medium storing program instructions for causing one or more processors to:
receive, from a distributed database service (DDS), usage data on resource request-units from a plurality of services that use resources provided by the DDS;
provide a plurality of datasets corresponding to a plurality of time windows, using the usage data;
transform at least a subset of the plurality of datasets, into at least three transformed datasets each corresponding to a different frequency range;
provide a predicted value of request-units according to the at least three transformed datasets; and
request the DDS to provide the resources up to a number of request-units at a given time instance according to the predicted value.
20. The non-transitory computer readable medium of claim 19 , wherein the program instructions further cause the one or more processors to:
transform the at least a subset of the plurality of datasets using Wavelet Transform, into at least three transformed datasets each corresponding to a different frequency range; and
provide a predicted value of request-units according to the at least three transformed datasets, using an autoregressive-moving average model (ARMA) method.