System and method for neural time series preprocessing
Systems and methods for neural time series preprocessing and forecasting, dividing time series data to generate chunks of short time series, inputting each of the short time series to a data preprocessing neural network that includes differencing to transform non-stationary data to stationary data and to filter noise, generating and outputting, from the data preprocessing neural network, processed time series data, and inputting the processed time series data to a forecasting neural network. Parameters of the data preprocessing neural network and parameters of the forecasting neural network are learned end-to-end.
1 . A system for preprocessing and forecasting neural time series data, the system comprising:
at least one processor; and
a memory comprising instructions which, when executed by the processor, configure the processor to:
generate a plurality of short time series data from a non-stationary time series data;
preprocess each of the plurality of short time series data using a data preprocessing neural network to generate a respective processed time series data, the data preprocessing neural network comprising a hyper neural network configured to receive the plurality of short time series data to dynamically generate preprocessing weight parameters which vary for each of the plurality of short time series data, the dynamically generated preprocessing weight parameters are weights utilized by the data preprocessing neural network to transform non-stationary data to more stationary data using a differencing transformation, the differencing transformation including computing differences between consecutive observations in the non-stationary data;
determine parameters of a forecasting neural network by optimizing a differentiable loss function based on output of the data preprocessing neural network, wherein parameters of the hyper neural network of the data preprocessing neural network and the parameters of the forecasting neural network are learned together in an end-to-end training process;
input the plurality of the respective processed time series data to the forecasting neural network to predict a next data point for the non-stationary time series data.
2 . The system as claimed in claim 1 , wherein the preprocessing weight parameters comprise:
a trained fixed component learned from the preprocessing neural network; and
a dynamic component comprising the output of the hyper neural network.
3 . The system as claimed in claim 1 , wherein the at least one processor is configured to determine the parameters of the forecasting neural network via stochastic gradient descent or derivatives of the stochastic gradient descent.
4 . The system as claimed in claim 1 , wherein the parameters are outputs by a neural network and serve as short term memories that depend on the particular input time series.
5 . The system as claimed in claim 1 , wherein the data preprocessing neural network is configured to estimate moving weighted averages for each of the plurality of short time series data.
6 . The system as claimed in claim 5 , wherein the moving weighted averages comprise at least one of: moving mean, or moving standard deviation.
7 . The system as claimed in claim 1 , wherein the data preprocessing neural network is configured to perform local regression for each of the plurality of short time series data.
8 . The system as claimed in claim 1 , wherein the differencing technique is generalized from a kernel scheme or an autoregressive moving average (ARIMA)-based differencing scheme.
9 . The system as claimed in claim 1 , wherein the preprocessing of the plurality of short time series data is optimized for forecasting using the forecasting neural network.
10 . The system as claimed in claim 1 , wherein the forecasting neural network comprises a recurrent neural net (RNN), a long short-term memory (LSTM), or a temporal convolutional neural networks (CNN).
11 . A computer-implemented method for neural time series preprocessing and forecasting, the method comprising:
generating a plurality of short time series data from a non-stationary time series data;
preprocessing each of the plurality of short time series data using a data preprocessing neural network to generate a respective processed time series data, the data preprocessing neural network comprising a hyper neural network configured to receive the plurality of short time series data to dynamically generate preprocessing weight parameters which vary for each of the plurality of short time series data, the dynamically generated preprocessing weight parameters are weights utilized by the data preprocessing neural network to transform non-stationary data to more stationary data using a differencing transformation including computing differences between consecutive observations in the non-stationary data;
determining parameters of a forecasting neural network by optimizing a differentiable loss function based on output of the data preprocessing neural network, wherein parameters of the hyper neural network of the data preprocessing neural network and the parameters of the forecasting neural network are learned together in an end-to-end training process;
inputting the plurality of the respective processed time series data to the forecasting neural network to predict a next data point for the non-stationary time series data.
12 . The method as claimed in claim 11 , wherein the preprocessing weight parameters comprise:
a trained fixed component learned from the preprocessing neural network; and
a dynamic component comprising the output of the hyper neural network.
13 . The method as claimed in claim 11 , comprising determining the parameters of the forecasting neural network via stochastic gradient descent or derivatives of the stochastic gradient descent.
14 . The method as claimed in claim 11 , wherein the parameters are outputs by a neural network and serve as short term memories that depend on the particular input time series.
15 . The method as claimed in claim 11 , wherein the data preprocessing neural network estimates moving weighted averages for each of the plurality of short time series data.
16 . The method as claimed in claim 15 , wherein the moving weighted averages comprise at least one of: moving mean, or moving standard deviation.
17 . The method as claimed in claim 11 , wherein the data preprocessing neural network performs local regression for each of the plurality of short time series data.
18 . The method as claimed in claim 11 , wherein the differencing technique is generalized from a kernel scheme or an autoregressive moving average (ARIMA)-based differencing scheme.
19 . The method as claimed in claim 11 , wherein the preprocessing of the plurality of short time series data is optimized for forecasting using the forecasting neural network.
20 . The method as claimed in claim 11 , wherein the forecasting neural network comprises a recurrent neural net (RNN), a long short-term memory (LSTM), or a temporal convolutional neural networks (CNN).