Systems and methods for aggregating time-series data streams based on potential state characteristics following aggregation
Systems and methods are described herein for novel uses and/or improvements to data aggregation related to artificial intelligence applications, specifically applications related to aggregating time-series data. As one example, systems and methods are described herein for predicting effects when aggregating time-series data and modifying the one or more data streams used to populate a model profile and/or feed an artificial intelligence application with the time-series data.
1. A system for aggregating time-series data streams based on potential state characteristics following aggregation, the system comprising:
one or more processors; and
one or more non-transitory machine readable media comprising instructions recorded thereon that when executed by the one or more processors cause operations comprising:
receiving a first model profile, wherein the first model profile is populated based on a first plurality of time-series data streams, and wherein the first model profile corresponds to a required state characteristic;
determining a first feature input based on the first model profile;
receiving a plurality of available time-series data streams;
generating training data comprising labeled feature inputs with known clusters using a historic data set having predetermined characteristics at a beginning or ending of a selected time period and a synthetically generated data set, wherein the synthetically generated data set is generated by resampling outlier events in the historic data set;
training, using the training data, a first neural network to classify labeled feature inputs with known clusters based on similarities between state characteristics in the labeled feature inputs and the known clusters;
using the first neural network to cluster the plurality of available time-series data streams into a first plurality of time-series data stream clusters by aggregating a subset of the plurality of available time-series data streams;
generating a synthetic profile corresponding to the first model profile;
retrieving the first plurality of time-series data stream clusters from the synthetic profile, wherein the first plurality of time-series data stream clusters is generated by the first neural network;
determining a second feature input based on the first plurality of time-series data stream clusters;
generating a plurality of respective feature inputs corresponding to each of the first plurality of time-series data stream clusters;
comparing the second feature input to the plurality of respective feature inputs;
determining, based on comparing the second feature input to the plurality of respective feature inputs, a first similarity metric between the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters;
selecting a first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric exceeding a first similarity threshold;
generating a second time-series data stream cluster based on aggregating the first time-series data stream cluster and the first plurality of time-series data streams;
determining a first state characteristic for the second time-series data stream cluster;
comparing the first state characteristic to the required state characteristic; and
generating, at a user interface, a first recommendation for the first time-series data stream cluster based on comparing the first state characteristic to the required state characteristic.
2. A method for aggregating time-series data streams based on potential state characteristics following aggregation, the method comprising:
receiving a first model profile, wherein the first model profile is populated based on a first plurality of time-series data streams, and wherein the first model profile corresponds to a required state characteristic;
determining a first feature input based on the first model profile;
receiving a plurality of available time-series data streams;
generating training data comprising labeled feature inputs with known clusters using a historic data set having predetermined characteristics at a beginning or ending of a selected time period and a synthetically generated data set, wherein the synthetically generated data set is generated by resampling outlier events in the historic data set;
training, using the training data, a first neural network to classify the labeled feature inputs with the known clusters based on similarities between state characteristics in the labeled feature inputs and the known clusters;
using the neural network to cluster the plurality of available time-series data streams into a first plurality of time-series data stream clusters by aggregating a subset of the plurality of available time-series data streams;
retrieving the first plurality of time-series data stream clusters, wherein the first plurality of time-series data stream clusters is generated by the first neural network;
determining a second feature input based on the first plurality of time-series data stream clusters;
determining, based on the first feature input and the second feature input, a first similarity metric between the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters;
selecting a first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric exceeding a first similarity threshold;
generating a second time-series data stream cluster based on aggregating the first time-series data stream cluster and the first plurality of time-series data streams;
determining a first state characteristic for the second time-series data stream cluster;
comparing the first state characteristic to the required state characteristic; and
generating, at a user interface, a first recommendation for the first time-series data stream cluster based on comparing the first state characteristic to the required state characteristic.
3. The method of claim 2 , wherein determining the first feature input based on the first model profile further comprises:
determining a vector array type corresponding to the first model profile; and
determining the first feature input based on the vector array type.
4. The method of claim 2 , wherein populating the first model profile based on the first plurality of time-series data streams further comprises:
receiving a first time stamp; and
determining respective values for each data stream of the first plurality of time-series data streams corresponding to the first time stamp, wherein each data stream comprises a plurality of values corresponding to different time stamps.
5. The method of claim 2 , wherein determining, based on the first feature input and the second feature input, the first similarity metric between the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters further comprises:
retrieving a first criterion for determining the first similarity metric, wherein the first criterion is based on the first model profile;
retrieving a second criterion for determining the first similarity metric, wherein the second criterion is based on the required state characteristic; and
performing a multivariable analysis of the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters based on the first criterion and the second criterion.
6. The method of claim 2 , wherein determining, based on the first feature input and the second feature input, the first similarity metric between the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters further comprises:
selecting an algorithm from a plurality of algorithms for processing the first feature input and the second feature input;
selecting a second artificial intelligence model from a plurality of artificial intelligence models based on the algorithm; and
inputting the first feature input and the second feature input into the second artificial intelligence model to generate a first output, wherein the first output comprises the first similarity metric.
7. The method of claim 2 , wherein determining, based on the first feature input and the second feature input, the first similarity metric between the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters further comprises:
generating a plurality of respective feature inputs corresponding to each of the first plurality of time-series data stream clusters; and
comparing the first feature input to the plurality of respective feature inputs.
8. The method of claim 2 , wherein determining the first state characteristic for the second time-series data stream cluster further comprises:
retrieving a normalization factor for the first plurality of time-series data streams; and
applying the normalization factor to the second time-series data stream cluster to generate the first state characteristic.
9. The method of claim 2 , wherein generating the second time-series data stream cluster based on aggregating the first time-series data stream cluster and the first plurality of time-series data streams further comprises:
retrieving a normalization factor for the first plurality of time-series data streams;
applying the normalization factor to the first time-series data stream cluster to generate a normalized time-series data stream cluster; and
aggregating the normalized time-series data stream cluster and the first plurality of time-series data streams to generate the second time-series data stream cluster.
10. The method of claim 2 , wherein selecting the first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric exceeding the first similarity threshold further comprises:
determining a profile characteristic for the first model profile; and
determining the first similarity threshold based on the profile characteristic.
11. The method of claim 2 , wherein selecting the first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric exceeding the first similarity threshold further comprises:
determining the first similarity threshold based on a minimum amount of similarity required between the first time-series data stream cluster from the first plurality of time-series data stream clusters and the first plurality of time-series data streams;
determining a second similarity threshold based on a maximum amount of similarity required between the first time-series data stream cluster from the first plurality of time-series data stream clusters and the first plurality of time-series data streams; and
selecting the first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric not exceeding the second similarity threshold.
12. The method of claim 2 , wherein generating the first recommendation for the first time-series data stream cluster based on comparing the first state characteristic to the required state characteristic further comprises:
determining a difference between the first state characteristic and the required state characteristic; and
determining whether the difference exceeds a threshold difference.
13. The method of claim 12 , wherein determining the difference between the first state characteristic and the required state characteristic further comprises:
determining a gradient for a time-series prediction based on the difference;
determining a magnitude of the gradient; and
identifying a predicted event for the time-series prediction based on the magnitude.
14. The method of claim 13 , further comprising:
determining an effect of the predicted event based on the magnitude; and
selecting the first recommendation from a plurality of recommendations based on the effect.
15. The method of claim 2 , wherein the first neural network is trained to cluster the plurality of available time-series data streams into the first plurality of time-series data stream clusters by aggregating the subset of the plurality of available time-series data streams based on correlation-distances between separate time-series data streams of the plurality of available time-series data streams.
16. The method of claim 2 , wherein training the first neural network comprises:
generating a matrix of pairwise correlations corresponding to the plurality of available time-series data streams; and
clustering the plurality of available time-series data streams based on pairwise distances.
17. One or more non-transitory machine readable media comprising instructions recorded thereon that when executed by one or more processors cause operations comprising:
receiving a first model profile, wherein the first model profile is populated based on a first plurality of time-series data streams, and wherein the first model profile corresponds to a required state characteristic;
receiving a plurality of available time-series data streams;
generating training data comprising labeled feature inputs with known clusters using a historic data set having predetermined characteristics at a beginning or ending of a selected time period and a synthetically generated data set, wherein the synthetically generated data set is generated by resampling outlier events in the historic data set;
training, using the training data, a first neural network to classify the labeled feature inputs with the known clusters based on similarities between state characteristics in the labeled feature inputs and the known clusters;
using the first neural network to cluster the plurality of available time-series data streams into a first plurality of time-series data stream clusters by aggregating a subset of the plurality of available time-series data streams;
retrieving the first plurality of time-series data stream clusters, wherein the first plurality of time-series data stream clusters is generated by the first neural network;
determining a first similarity metric between the first plurality of time-series data streams and each of the first plurality of time-series data stream clusters;
selecting a first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric exceeding a first similarity threshold;
generating a second time-series data stream cluster based on aggregating the first time-series data stream cluster and the first plurality of time-series data streams;
determining a first state characteristic for the second time-series data stream cluster;
comparing the first state characteristic to the required state characteristic; and
generating, at a user interface, a first recommendation for the first time-series data stream cluster based on comparing the first state characteristic to the required state characteristic.
18. The one or more non-transitory machine readable media of claim 17 , wherein populating the first model profile based on the first plurality of time-series data streams further comprises:
receiving a first time stamp; and
determining respective values for each data stream of the first plurality of time-series data streams corresponding to the first time stamp, wherein each of the data streams comprises a plurality of values corresponding to different time stamps.
19. The one or more non-transitory machine readable media of claim 17 , wherein selecting the first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric exceeding the first similarity threshold further comprises:
determining the first similarity threshold based on a minimum amount of similarity required between the first time-series data stream cluster from the first plurality of time-series data stream clusters;
determining a second similarity threshold based on a maximum amount of similarity required between the first time-series data stream cluster from the first plurality of time-series data stream clusters; and
selecting the first time-series data stream cluster from the first plurality of time-series data stream clusters based on the first similarity metric not exceeding the second similarity threshold.