Computing environment scaling
A system uses a machine learning model to identify anomalies and modify parameters of a computing environment. The system modifies parameters of a computing environment based on the presence and absence of anomalies in the computing system while avoiding modifying parameters as a result of brief spikes in computing environment attributes. The system uses a machine learning model to generate predictions of anomalies for data points of computing environment attributes. The system compiles sets of predictions into batches. The system determines whether each batch includes enough anomalous-labeled data points to be considered an anomalous batch. The system compiles the batches into sets. The system determines whether the sets of batches include enough anomalous batches to be considered an anomalous set of batches. The system modifies the parameters of the computing environment based on determining whether or not the sets of batches are anomalous.
1 . A non-transitory computer readable medium comprising instructions which, when executed by one or more hardware processors, causes performance of operations comprising:
monitoring a computing environment to obtain a data set, wherein the data set comprises a plurality of data points, each data point comprising a plurality of attributes of the computing environment, wherein the computing environment is configured with a first configuration of parameters for storing and processing data in the computing environment;
applying a machine learning model to a data point among the plurality of data points to generate a prediction whether the data point corresponds to an anomaly in the computing environment;
grouping sets of consecutively-generated data points into a plurality of batches, each batch corresponding to a different segment of time;
for each particular batch of the plurality of batches:
classifying the particular batch as anomalous or non-anomalous based on a number of data points in the particular batch that are predicted to be anomalous by the machine learning model;
analyzing a first set of consecutively-occurring batches from among the plurality of batches; and
based on determining that a number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets a second threshold number: modifying the first configuration of computing resources in the computing environment to configure the computing environment with a second configuration of computing resources at least by:
up-scaling or down-scaling the computing resources based on a predicted anomaly in the computing environment,
wherein up-scaling or down-scaling the computing resources based on the predicted anomaly includes at least one of: modifying a number of computing resources available to execute tasks in the computing environment, modifying a storage capacity of the computing resources, and modifying a data transmission capacity of the computing resources.
2 . The medium of claim 1 , wherein the operations further comprise:
obtaining historical data associated with historical computing environment attributes;
generate a training data set from the historical data, the training data set comprising:
historical data points comprising historical attribute data for the plurality of attributes of the computing environment, and
for each historical data point, a label indicating whether the historical data point is associated with the anomaly in the computing environment; and
training the machine learning model using the training data set to generate, for a particular data point of attribute data of the computing environment, a prediction whether the particular data point corresponds to an anomaly in the computing environment.
3 . The medium of claim 2 , wherein the training data set further comprises:
historical computing environment parameter data,
wherein the machine learning model is further trained using the trained data set to generate, for the particular data point of attribute data of the computing environment, a recommendation for modifying one or more computing environment parameters associated with the anomaly in the computing environment.
4 . The medium of claim 1 , wherein the second threshold number is a parameter-upscaling threshold number,
wherein modifying the first configuration of the computing resources in the computing environment comprises: up-scaling the computing resources proportional to a magnitude of a predicted anomaly in the computing environment.
5 . The medium of claim 4 , wherein the operations further comprise:
subsequent to up-scaling the computing resources in the computing environment:
detecting a predetermined period of time has elapsed;
during the predetermined period of time, analyzing a second set of consecutively-occurring batches from among the plurality of batches; and
based on determining that a number of batches identified as anomalous, from among the second set of consecutively-occurring batches, meets a parameter-downscaling threshold number: downscaling the computing resources in the computing environment.
6 . The medium of claim 1 , wherein the second threshold number is a computing-resource-downscaling threshold number,
wherein modifying the first configuration of computing resources in the computing environment comprises: down-scaling the computing resources in the computing environment.
7 . The medium of claim 1 , wherein the operations further comprise:
receiving user input selecting one action to perform based on determining that the number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets the second threshold number, the one action selected from among: (a) automatically modifying the first configuration of the computing resources in the computing environment, and (b) generating a notification indicating that the number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets the second threshold number.
8 . The medium of claim 1 , wherein determining that the number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets the second threshold number includes labeling the first set of consecutively-occurring batches as anomalous, and
wherein modifying the first configuration of the computing resources in the computing environment is based on determining the first set of consecutively-occurring batches is labeled as anomalous.
9 . The medium of claim 1 , wherein the computing environment is a cloud-based computing environment including a plurality of compute nodes and at least one intermediate node, the at least one intermediate node comprising a task queue for directing computing tasks to the plurality of compute nodes,
wherein the plurality of attributes of the computing environment includes a first set of attributes of the plurality of compute nodes and a second set of attributes of the at least one intermediate node,
wherein modifying the first configuration of the computing resources in the computing environment includes increasing a number of compute nodes available to the intermediate node in the computing environment for executing the computing tasks.
10 . The non-transitory computer readable medium of claim 1 , wherein the sets of consecutively-generated data points grouped into the plurality of batches comprise data points for which the machine learning model has generated a respective prediction whether the respective data point corresponds to an anomaly in the computing environment.
11 . The non-transitory computer readable medium of claim 1 , wherein up-scaling or down-scaling the computing resources comprises up-scaling or down-scaling at least one of:
a number of compute nodes in the computing environment;
a specific type of compute node in the computing environment;
a number of intermediate nodes in the computing environment;
a number of database nodes in the computing environment;
a size of an existing node in the computing environment;
a division of a partition of the existing node in the computing environment;
a processing capacity available in the computing environment;
a data storage capacity available in the computing environment;
a data transmission capacity available in the computing environment; and
an input/output (I/O) capacity available in the computing environment.
12 . The non-transitory computer readable medium of claim 1 , wherein monitoring the computing environment to obtain the data set comprises monitoring a set of compute nodes configured to execute computing tasks in the computing environment to determine values for a usage metric for the set of compute nodes,
wherein up-scaling or down-scaling the computing resources comprises at least one of:
adding a compute node to the set of compute nodes in the computing environment; and
increasing a number of processors available to the set of compute nodes for executing the computing tasks.
13 . The non-transitory computer readable medium of claim 1 , the up-scaling or down-scaling the computing resources is based on a magnitude of the predicted anomaly in the computing environment.
14 . A method comprising:
monitoring a computing environment to obtain a data set, wherein the data set comprises a plurality of data points, each data point comprising a plurality of attributes of the computing environment, wherein the computing environment is configured with a first configuration of parameters for storing and processing data in the computing environment;
applying a machine learning model to a data point among the plurality of data points to generate a prediction whether the data point corresponds to an anomaly in the computing environment;
grouping sets of consecutively-generated data points into a plurality of batches, each batch corresponding to a different segment of time;
for each particular batch of the plurality of batches:
classifying the particular batch as anomalous or non-anomalous based on a number of data points in the particular batch that are predicted to be anomalous by the machine learning model;
analyzing a first set of consecutively-occurring batches from among the plurality of batches; and
based on determining that a number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets a second threshold number: modifying the first configuration of computing resources in the computing environment to configure the computing environment with a second configuration of computing resources at least by:
up-scaling or down-scaling the computing resources based on a predicted anomaly in the computing environment,
wherein up-scaling or down-scaling the computing resources based on the predicted anomaly includes at least one of: modifying a number of computing resources available to execute tasks in the computing environment, modifying a storage capacity of the computing resources, and modifying a data transmission capacity of the computing resources.
15 . The method of claim 14 , further comprising:
obtaining historical data associated with historical computing environment attributes;
generate a training data set from the historical data, the training data set comprising:
historical data points comprising historical attribute data for the plurality of attributes of the computing environment, and
for each historical data point, a label indicating whether the historical data point is associated with the anomaly in the computing environment; and
training the machine learning model using the training data set to generate, for a particular data point of attribute data of the computing environment, a prediction whether the particular data point corresponds to an anomaly in the computing environment.
16 . The method of claim 15 , wherein the training data set further comprises:
historical computing environment parameter data,
wherein the machine learning model is further trained using the trained data set to generate, for the particular data point of attribute data of the computing environment, a recommendation for modifying one or more computing environment parameters associated with the anomaly in the computing environment.
17 . The method of claim 14 , wherein the second threshold number is a parameter-upscaling threshold number,
wherein modifying the first configuration of the computing resources in the computing environment comprises: up-scaling the computing resources proportional to a magnitude of a predicted anomaly in the computing environment.
18 . The method of claim 17 , further comprising:
subsequent to up-scaling the computing resources in the computing environment:
detecting a predetermined period of time has elapsed;
during the predetermined period of time, analyzing a second set of consecutively-occurring batches from among the plurality of batches; and
based on determining that a number of batches identified as anomalous, from among the second set of consecutively-occurring batches, meets a parameter-downscaling threshold number: downscaling the computing resources in the computing environment.
19 . The method of claim 14 , wherein the second threshold number is a computing-resource-downscaling threshold number,
wherein modifying the first configuration of computing resources in the computing environment comprises: down-scaling the computing resources in the computing environment.
20 . The method of claim 14 , further comprising:
receiving user input selecting one action to perform based on determining that the number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets the second threshold number, the one action selected from among: (a) automatically modifying the first configuration of the computing resources in the computing environment, and (b) generating a notification indicating that the number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets the second threshold number.
21 . The method of claim 14 , wherein determining that the number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets the second threshold number includes labeling the first set of consecutively-occurring batches as anomalous, and
wherein modifying the first configuration of the computing resources in the computing environment is based on determining the first set of consecutively-occurring batches is labeled as anomalous.
22 . The method of claim 14 , wherein the computing environment is a cloud-based computing environment including a plurality of compute nodes and at least one intermediate node, the at least one intermediate node comprising a task queue for directing computing tasks to the plurality of compute nodes,
wherein the plurality of attributes of the computing environment includes a first set of attributes of the plurality of compute nodes and a second set of attributes of the at least one intermediate node,
wherein modifying the first configuration of the computing resources in the computing environment includes increasing a number of compute nodes available to the intermediate node in the computing environment for executing the computing tasks.
23 . A system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to perform:
monitoring a computing environment to obtain a data set, wherein the data set comprises a plurality of data points, each data point comprising a plurality of attributes of the computing environment, wherein the computing environment is configured with a first configuration of parameters for storing and processing data in the computing environment;
applying a machine learning model to a data point among the plurality of data points to generate a prediction whether the data point corresponds to an anomaly in the computing environment;
grouping sets of consecutively-generated data points into a plurality of batches, each batch corresponding to a different segment of time;
for each particular batch of the plurality of batches:
classifying the particular batch as anomalous or non-anomalous based on a number of data points in the particular batch that are predicted to be anomalous by the machine learning model;
analyzing a first set of consecutively-occurring batches from among the plurality of batches; and
based on determining that a number of batches identified as anomalous, from among the first set of consecutively-occurring batches, meets a second threshold number: modifying the first configuration of computing resources in the computing environment to configure the computing environment with a second configuration of computing resources at least by:
up-scaling or down-scaling the computing resources based on a predicted anomaly in the computing environment,
wherein up-scaling or down-scaling the computing resources based on the predicted anomaly includes at least one of: modifying a number of computing resources available to execute tasks in the computing environment, modifying a storage capacity of the computing resources, and modifying a data transmission capacity of the computing resources.
24 . The system of claim 23 , wherein the instructions further cause:
obtaining historical data associated with historical computing environment attributes;
generate a training data set from the historical data, the training data set comprising:
historical data points comprising historical attribute data for the plurality of attributes of the computing environment, and
for each historical data point, a label indicating whether the historical data point is associated with the anomaly in the computing environment; and
training the machine learning model using the training data set to generate, for a particular data point of attribute data of the computing environment, a prediction whether the particular data point corresponds to an anomaly in the computing environment.
25 . The non-transitory computer readable medium of claim 13 , wherein the operations further comprise:
determining the magnitude of the predicted anomaly in the computing environment based on the number of batches identified as anomalous.