Method for forecasting distributed resource utilization in a virtualization environment
A method for time series analysis of time-oriented usage data pertaining to computing resources of a computing system. A method embodiment commences upon collecting time series datasets, individual ones of the time series datasets comprising time-oriented usage data of a respective individual computing resource. A plurality of prediction models are trained using portions of time-oriented data. The trained models are evaluated to determine quantitative measures pertaining to predictive accuracy. One of the trained models is selected and then applied over another time series dataset of the individual resource to generate a plurality of individual resource usage predictions. The individual resource usage predictions are used to calculate seasonally-adjusted resource usage demand amounts over a future time period. The resource usage demand amounts are compared to availability of the resource to form a runway that refers to a future time period when the resource is predicted to be demanded to its capacity.
1. A computer-implemented method, comprising:
storing, in a database associated with a virtualization environment, usage data for a plurality of users corresponding to respective nodes of a plurality of nodes in the virtualization environment, wherein the usage data corresponds to a distributed computer resource that comprises a storage pool, the usage data represents usage of storage of the storage pool by respective users of the plurality of users, and the storage pool is accessed through a plurality of virtualized controllers;
determining computer resource demand for the distributed computer resource in the virtualization environment at least by:
training a plurality of prediction models, wherein each individual prediction model of the plurality of prediction models is trained at least by processing a subset of the usage data stored in the database associated with the virtualization environment corresponding to a respective user of the plurality of users, multiple predictions models of the plurality of prediction models reflect seasonality of usage of the storage of the storage pool by multiple users of the plurality of users over time;
generating a plurality of predictions using the plurality of trained prediction models, wherein individual predictions of the plurality of predictions correspond to respective individual users and represent predicted demand from the respective individual users, and the plurality of predictions reflect the seasonality of usage of the storage of the storage pool by the multiple users over time;
aggregating the plurality of predictions into an aggregated prediction that represents aggregate predicted demand of the storage of the storage pool accessed through the plurality of virtualized controllers, and the aggregated prediction represents seasonally-adjusted usage of the storage of the storage pool by the plurality of users over time; and
wherein the determined computer resource demand is based at least in part upon the aggregated predictions; and
generating a graphical user interface having at least a representation of the determined computer resource demand.
2. The computer-implemented method of claim 1 , wherein a first prediction of the plurality of predictions comprises a first time series of values, a second prediction of the plurality of predictions comprises a second time series of values, the aggregated prediction comprises a combination of the first time series and the second time series, and the method further comprises identifying when the distributed computer resource is predicted to be depleted at least by comparing the aggregated prediction to the distributed computer resource.
3. The computer-implemented method of claim 1 , wherein virtualized controllers of the plurality of virtualized controllers are on individual nodes of a plurality of nodes in the virtualization environment, and the plurality of virtualized controllers present storage local to a corresponding node as part of the storage pool.
4. The computer-implemented method of claim 1 , wherein the representation of the determined computer resource demand comprises a runway view representing when demand is predicted to meet a threshold corresponding to computer resource capacity.
5. The computer-implemented method of claim 4 , wherein aggregating a first prediction and a second prediction of the plurality of predictions into the aggregated prediction comprises applying a respective weight to results from at least one of a first or second prediction model of the plurality of prediction models, the determined computer resource demand pertains to a storage container, the runway view includes a first resource runway and a second resource runway, the first resource runway is indicative of a first future timepoint or a first future temporal period when the distributed computer resource is predicted to be demanded to a threshold capacity of the distributed computer resource, and the second resource runway is indicative of a second future timepoint or a second future temporal period when the distributed computer resource is predicted to be demanded to a second threshold capacity of the distributed computer resource.
6. The computer-implemented method of claim 1 , wherein the plurality of prediction models comprise at least one of an ARIMA (autoregressive integrated moving average) model, an ETS (exponential smooth) model, an STL (seasonal trend decomposition using Loess) model, a NEURAL NETWORK model, a RANDOM WALK model, a SEASONAL NAÏVE model, a MEAN model, or a LINEAR REGRESSION WITH SEASONAL COMPONENTS model, or a combination thereof.
7. The computer-implemented method of claim 1 , wherein two or more nodes of the plurality of nodes generate the plurality of predictions.
8. The computer-implemented method of claim 7 , wherein the determined computer resource demand pertains to a storage container that comprises an empty virtual disk, a virtual disk being populated with some actual data, a file or object, or a storage space for a virtual machine and the storage container comprises an area of storage in the storage pool that is managed by a controller virtual machine above a hypervisor executing on a node in the virtualization environment.
9. The computer-implemented method of claim 1 , further comprises determining another aggregated prediction based on a system level prediction using a system level prediction model generated by processing aggregated usage data, wherein the aggregated usage data is generated by aggregating the usage data of the respective users for corresponding time periods, and wherein the virtualization environment comprises a first cluster, a second virtualization environment comprises a second cluster, and determining a second computer resource demand for a second distributed computer resource in the second virtualization environment.
10. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by a processor, causes the processor to perform a set of acts, the set of acts comprising:
storing, in a database associated with a virtualization environment, usage data for a plurality of users corresponding to respective nodes of a plurality of nodes in the virtualization environment, wherein the usage data corresponds to a distributed computer resource that comprises a storage pool, the usage data represents usage of storage of the storage pool by respective users of the plurality of users, and the storage pool is accessed through a plurality of virtualized controllers;
determining computer resource demand for the distributed computer resource in the virtualization environment at least by:
training a plurality of prediction models, wherein each individual prediction model of the plurality of prediction models is trained at least by processing a subset of the usage data stored in the database associated with the virtualization environment corresponding to a respective user of the plurality of users, multiple predictions models of the plurality of prediction models reflect seasonality of usage of the storage of the storage pool by multiple users of the plurality of users over time;
generating a plurality of predictions using the plurality of trained prediction models, wherein individual predictions of the plurality of predictions correspond to respective individual users and represent predicted demand from the respective individual users, and the plurality of predictions reflect the seasonality of usage of the storage of the storage pool by the multiple users over time;
aggregating the plurality of predictions into an aggregated prediction that represents aggregate predicted demand of the storage of the storage pool accessed through the plurality of virtualized controllers, and the aggregated prediction represents seasonally-adjusted usage of the storage of the storage pool by the plurality of users over time; and
wherein the determined computer resource demand is based at least in part upon the aggregated predictions; and
generating a graphical user interface having at least a representation of the determined computer resource demand.
11. The non-transitory computer readable medium of claim 10 , wherein a first prediction of the plurality of predictions comprises a first time series of values, a second prediction of the plurality of predictions comprises a second time series of values, the aggregated prediction comprises a combination of the first time series and the second time series, and wherein the set of acts further comprise identifying when the distributed computer resource is predicted to be depleted at least by comparing the aggregated prediction to the distributed computer resource.
12. The non-transitory computer readable medium of claim 10 , wherein virtualized controllers of the plurality of virtualized controllers are on individual nodes of a plurality of nodes in the virtualization environment, and the plurality of virtualized controllers present storage local to a corresponding node as part of the storage pool.
13. The non-transitory computer readable medium of claim 10 , wherein the representation of the determined computer resource demand comprises a runway view representing when demand is predicted to meet a threshold corresponding to computer resource capacity.
14. The non-transitory computer readable medium of claim 13 , wherein aggregating a first prediction and a second prediction of the plurality of predictions into the aggregated prediction comprises applying a respective weight to results from at least one of a first or second prediction model of the plurality of prediction models, the determined computer resource demand pertains to a storage container, the runway view includes a first resource runway and a second resource runway, the first resource runway is indicative of a first future timepoint or a first future temporal period when the distributed computer resource is predicted to be demanded to a threshold capacity of the distributed computer resource, and the second resource runway is indicative of a second future timepoint or a second future temporal period when the distributed computer resource is predicted to be demanded to a second threshold capacity of the distributed computer resource.
15. A computing system, comprising:
a processor; and
a storage medium having stored thereon a sequence of instructions which when executed by the processor, causes the processor to execute a set of acts, the set of acts comprising:
storing, in a database associated with a virtualization environment, usage data for a plurality of users corresponding to respective nodes of a plurality of nodes in the virtualization environment, wherein the usage data corresponds to a distributed computer resource that comprises a storage pool, the usage data represents usage of storage of the storage pool by respective users of the plurality of users, and the storage pool is accessed through a plurality of virtualized controllers;
determining computer resource demand for the distributed computer resource in the virtualization environment at least by:
training a plurality of prediction models, wherein each individual prediction model of the plurality of prediction models is trained at least by processing a subset of the usage data stored in the database associated with the virtualization environment corresponding to a respective user of the plurality of users, multiple predictions models of the plurality of prediction models reflect seasonality of usage of the storage of the storage pool by multiple users of the plurality of users over time;
generating a plurality of predictions using the plurality of trained prediction models, wherein individual predictions of the plurality of predictions correspond to respective individual users and represent predicted demand from the respective individual users, and the plurality of predictions reflect the seasonality of usage of the storage of the storage pool by the multiple users over time;
aggregating the plurality of predictions into an aggregated prediction that represents aggregate predicted demand of the storage of the storage pool accessed through the plurality of virtualized controllers, and the aggregated prediction represents seasonally-adjusted usage of the storage of the storage pool by the plurality of users over time; and
wherein the determined computer resource demand is based at least in part upon the aggregated predictions; and
generating a graphical user interface having at least a representation of the determined computer resource demand.
16. The computing system of claim 15 , wherein a first prediction of the plurality of predictions comprises a first time series of values, a second prediction of the plurality of predictions comprises a second time series of values, the aggregated prediction comprises a combination of the first time series and the second time series, and wherein the set of acts further comprise identifying when the distributed computer resource is predicted to be depleted at least by comparing the aggregated prediction to the distributed computer resource.
17. The computing system of claim 15 , wherein the determined computer resource demand pertains to a storage container that comprises an empty virtual disk, a virtual disk being populated with some actual data, a file or object, or a storage space for a virtual machine.
18. The computing system of claim 15 , wherein the representation of the determined computer resource demand comprises a runway view representing when demand is predicted to meet a threshold corresponding to computer resource capacity.
19. The computing system of claim 18 , wherein aggregating a first prediction and a second prediction of the plurality of predictions into the aggregated prediction comprises applying a respective weight to results from at least one of a first or second prediction model of the plurality of prediction models, the determined computer resource demand pertains to a storage container, the runway view includes a first resource runway and a second resource runway, the first resource runway is indicative of a first future timepoint or a first future temporal period when the distributed computer resource is predicted to be demanded to a threshold capacity of the distributed computer resource, and the second resource runway is indicative of a second future timepoint or a second future temporal period when the distributed computer resource is predicted to be demanded to a second threshold capacity of the distributed computer resource.
20. The computing system of claim 15 , wherein the plurality of prediction models comprise at least one of an ARIMA (autoregressive integrated moving average) model, an ETS (exponential smooth) model, an STL (seasonal trend decomposition using Loess) model, a NEURAL NETWORK model, a RANDOM WALK model, a SEASONAL NAÏVE model, a MEAN model, or a LINEAR REGRESSION WITH SEASONAL COMPONENTS model, or a combination thereof.
21. The non-transitory computer readable medium of claim 10 , wherein the plurality of prediction models comprise at least one of an ARIMA (autoregressive integrated moving average) model, an ETS (exponential smooth) model, an STL (seasonal trend decomposition using Loess) model, a NEURAL NETWORK model, a RANDOM WALK model, a SEASONAL NAÏVE model, a MEAN model, or a LINEAR REGRESSION WITH SEASONAL COMPONENTS model, or a combination thereof.
22. The non-transitory computer readable medium of claim 10 , wherein two or more nodes of the plurality of nodes generate the plurality of predictions.
23. The non-transitory computer readable medium of claim 22 , wherein the determined computer resource demand pertains to a storage container that comprises an empty virtual disk, a virtual disk being populated with some actual data, a file or object, or a storage space for a virtual machine and the storage container comprises an area of the storage in the storage pool that is managed by a controller virtual machine above a hypervisor executing on a node in the virtualization environment.
24. The non-transitory computer readable medium of claim 10 , wherein the set of acts further comprise determining another aggregated prediction based on a system level prediction using a system level prediction model generated by processing aggregated usage data, wherein the aggregated usage data is generated by aggregating the usage data of the respective users for corresponding time periods, and wherein the virtualization environment comprises a first cluster, a second virtualization environment comprises a second cluster, and determining a second computer resource demand for a second distributed computer resource in the second virtualization environment.