MULTI-API METRIC MODELING USING LSTM SYSTEM
The present system models multiple application program interfaces(APIs) and determines anomaly behavior for the group of APIs. The system APIs are monitored and data is collected for the multiple APIs. Metrics are generated for the APIs and reported to an application. The metrics are a raw timeseries stream of metrics and are transformed to a different domain for processing. In some instances, the raw time series metric data is smoothed or averaged into an average domain. A model receives the smooth time series metric data, a pilot signal, and homogeneous signal inputs. The model may include an LSTM model or some other model. The LSTM model may output data to a neural network, which then provides output of a predicted value of the metrics, current value of the metric, and a regenerated pilot signal. A determination is made as to whether the neural network system predicts the pilot signal correctly, and if so the predicted metric is compared to the actual metric. If the metrics do not match, a degree of anomaly is determined and reported. In some instances, whether an anomaly is reported to the group of APIs depends on how many anomalies are detected in the severity of each anomaly.
1 . A method for forecasting values for system metric time series, comprising:
receiving raw time series metric data by an application on an application server, the raw time series metric data associated with an API provided by a remote server, the raw time metric data associated with request and responses to the API from a plurality of remote devices in communication with the remote server;
transforming, by the application on the application server, the received raw time series metric data into smoothed time series metric data;
passing the smoothed time series metric data through a prediction model, the prediction model trained using training data prior to receiving smoothed time series metric data;
passing one or more homogeneous variables associated with the API through the prediction model;
generating a prediction for the time series metric data in a smoothed format, the prediction based on the smoothed time series metric data and the one or more homogeneous variables associated with the API;
transforming the smoothed predicted time series metric data into a raw predicted time series metric value; and
analyzing the raw predicted time series metric value to determine whether raw time series metric data is anomalous.
2 . The method of claim 1 , wherein the raw time series metric data is associated with a plurality of APIs.
3 . The method of claim 2 , wherein the determination whether the raw time series metric data is anomalous is made based on the predicted value for each of a majority of the plurality of APIs.
4 . The method of claim 1 , further comprising:
receiving a pilot signal to identify a particular API of the plurality of APIs; and
generating a prediction of the pilot signal based on the output of the prediction model.
5 . The method of claim 1 , further comprising:
passing the output of the prediction model through a neural network, the generated prediction for the time series metric data and the generated prediction for the pilot signal being generated by the neural network.
6 . The method of claim 1 , wherein transforming includes using a moving average of the metric time series.
7 . The method of claim 1 , wherein the prediction model includes a long short-term memory model.
8 . A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for automatically forecasting values for system metric time series, the method comprising:
receiving raw time series metric data by an application on an application server, the raw time series metric data associated with an API provided by a remote server, the raw time metric data associated with request and responses to the API from a plurality of remote devices in communication with the remote server;
transforming, by the application on the application server, the received raw time series metric data into smoothed time series metric data;
passing the smoothed time series metric data through a prediction model, the prediction model trained using training data prior to receiving smoothed time series metric data;
passing one or more homogeneous variables associated with the API through the prediction model;
generating a prediction for the time series metric data in a smoothed format, the prediction based on the smoothed time series metric data and the one or more homogeneous variables associated with the API;
transforming the smoothed predicted time series metric data into a raw predicted time series metric value; and
analyzing the raw predicted time series metric value to determine whether raw time series metric data is anomalous.
9 . The non-transitory computer readable storage medium of claim 8 , wherein the raw time series metric data is associated with a plurality of APIs.
10 . The non-transitory computer readable storage medium of claim 9 , wherein the determination whether the raw time series metric data is anomalous is made based on the predicted value for each of a majority of the plurality of APIs.
11 . The non-transitory computer readable storage medium of claim 8 , the method further comprising:
receiving a pilot signal to identify a particular API of the plurality of APIs; and
generating a prediction of the pilot signal based on the output of the prediction model.
12 . The non-transitory computer readable storage medium of clam 8 , the method further comprising:
passing the output of the prediction model through a neural network, the generated prediction for the time series metric data and the generated prediction for the pilot signal being generated by the neural network.
13 . The non-transitory computer readable storage medium of claim 8 , wherein transforming includes using a moving average of the metric time series.
14 . The non-transitory computer readable storage medium of claim 8 , wherein the prediction model includes a long short-term memory model.
15 . A system for automatically forecasting values for system metric time series, comprising:
a server including a memory and a processor; and
one or more modules stored in the memory and executed by the processor to receive raw time series metric data by an application on an application server, the raw time series metric data associated with an API provided by a remote server, the raw time metric data associated with request and responses to the API from a plurality of remote devices in communication with the remote server, transform, by the application on the application server, the received raw time series metric data into smoothed time series metric data, pass the smoothed time series metric data through a prediction model, the prediction model trained using training data prior to receiving smoothed time series metric data, pass one or more homogeneous variables associated with the API through the prediction model, generate a prediction for the time series metric data in a smoothed format, the prediction based on the smoothed time series metric data and the one or more homogeneous variables associated with the API, transform the smoothed predicted time series metric data into a raw predicted time series metric value, and analyze the raw predicted time series metric value to determine whether raw time series metric data is anomalous.
16 . The system of claim 15 , wherein the raw time series metric data is associated with a plurality of APIs.
17 . The system of claim 16 , wherein the determination whether the raw time series metric data is anomalous is made based on the predicted value for each of a majority of the plurality of APIs.
18 . The system of claim 15 , the modules further executable to receive a pilot signal to identify a particular API of the plurality of APIs, and generate a prediction of the pilot signal based on the output of the prediction model.
19 . The system of claim 15 , the modules further executable to pass the output of the prediction model through a neural network, the generated prediction for the time series metric data and the generated prediction for the pilot signal being generated by the neural network.
20 . The system of claim 15 , wherein transforming includes using a moving average of the metric time series.