Generating input data for a machine learning model
A computer-implemented method includes, for each of a set of training dates: receiving, for each of a sequence of dates including the training date, an input data array representing values of a predetermined set of date-dependent features; receiving a target output corresponding to an evaluation of a predetermined metric at the training date; and performing an update routine including processing the input data array for each date using first layers of a neural network, processing a resulting intermediate data array using second layers of the neural network to generate a network output, and updating values of parameters of the neural network in in a direction of a negative gradient of an error between the target output and the network output. The data processing system is then arranged to generate an embedding array by processing an input data array for each of a given sequence of dates using the first layers of the neural network.
1. A computer-implemented method comprising:
training a data processing system to generate a plurality of embedding arrays for a set of target dates, each of the plurality of embedding arrays having a first number of dimensions and representing information associated with a respective target date of the set of target dates, the training comprising, for each sequence of dates including training dates from a set of training dates:
receiving, for each training date of a respective sequence of dates, a respective input data array, wherein each respective input data array is received sequentially in a chronological order based on the training dates of the respective sequence of dates, each respective input data array having a second number of dimensions greater than the first number of dimensions and representing values of a predetermined set of date-dependent features;
receiving a target output value corresponding to an evaluation of a predetermined metric for each training date; and
performing an update routine comprising:
processing each respective input data array sequentially received for each training date in the respective sequence of dates using first one or more layers of a neural network to generate an intermediate data array for a last training date in the respective sequence of dates, wherein the intermediate data array has the first number of dimensions,
processing the intermediate data array using second one or more layers of the neural network to generate a network output value,
determining an error between the network output value and the target output value for the last training date in the respective sequence of dates, and
updating values of a set of parameters of the neural network in a direction of a negative gradient of the determined error between the network output value and the target output value,
wherein, when the update routine has been performed for each training date of the set of training dates, the trained first one or more layers are obtained, and
wherein the computer-implemented method further comprises:
generating, by the data processing system, the plurality of embedding arrays using the trained first one or more layers, by processing respective target input data arrays, each respective target input data array corresponding to the respective target date of the set of target dates, wherein, as a result of the processing by the trained first one or more layers, each of the plurality of embedding arrays that corresponds to the respective target date is dependent on the respective input data array for the target date and dates other than the target date in the set of target dates; and
transmitting the plurality of embedding arrays to a remote computing system over a network, for training a forecasting model.
2. The computer-implemented method of claim 1 , wherein the first one or more layers of the neural network include one or more recurrent neural network layers.
3. The computer-implemented method of claim 2 , wherein the first one or more layers of the neural network include at least one long short-term memory unit.
4. The computer-implemented method of claim 1 , wherein the respective target date is a training date of the set of training dates.
5. The computer-implemented method of claim 1 , wherein for each training date of the set of training dates, the target output value corresponds to an evaluation of a financial metric at the training date.
6. The computer-implemented method of claim 1 , wherein the
plurality of embedding arrays are generated using respective different target output values corresponding to evaluations of respective different predetermined metrics at the training date, the computer-implemented method further comprising:
combining the plurality of embedding arrays for the respective target date into a combined embedding array for the respective target date.
7. The computer-implemented method of claim 1 , further comprising processing the plurality of embedding arrays, using supervised learning, to train the forecasting model to predict values of a date-dependent variable on a given target date, based on inputs including an embedding array for a given target date.
8. The computer-implemented method of claim 1 , further comprising, at the remote computing system:
receiving the plurality of embedding arrays; and
receiving a set of training values corresponding to evaluations of a forecasting metric at past dates corresponding to a subset of the plurality of embedding arrays;
processing the set of training dates and said subset of the plurality of embedding arrays using supervised learning to train the forecasting model to predict values of the forecasting metric on a given target date based on input data including an embedding array for the given target date; and
processing one or more of the plurality of embedding arrays using the trained forecasting model to predict values of the forecasting metric on dates corresponding to said one or more of the plurality of embedding arrays.
9. A data processing system comprising processing circuitry and memory circuitry, wherein the memory circuitry holds machine readable instructions which, when executed by the processing circuitry, cause the data processing system to execute a method including:
performing processing to generate a plurality of embedding arrays for a set of target dates, each of the plurality of embedding arrays having a first number of dimensions and representing information associated with a respective target date of the set of target dates, the performing the processing including, for each sequence of dates including training dates from a set of training dates:
receiving, for each training date of a respective sequence of dates, a respective input data array, wherein each respective input data array is received sequentially in a chronological order based on the training dates of the respective sequence of dates, each respective input data array having a second number of dimensions greater than the first number of dimensions and representing values of a predetermined set of date-dependent features;
receiving a target output value corresponding to an evaluation of a predetermined metric for each training date; and
performing an update routine including:
processing each respective input data array sequentially received for each training date in the respective sequence of dates using first one or more layers of a neural network to generate an intermediate data array for a last training date in the respective sequence of dates, wherein the intermediate data array has the first number of dimensions,
processing the intermediate data array using second one or more layers of the neural network to generate a network output value;
determining an error between the network output value and the target output value; for the last training date in the respective sequence of dates, and
updating values of a set of parameters of the neural network in a direction of a negative gradient of the determined error between the network output value and the target output value,
wherein, when the update routine has been performed for each training date of the set of training dates, the trained first one or more layers are obtained, and
wherein the method further includes:
generating the plurality of embedding arrays using the trained first one or more layers, by processing respective target input data arrays, each respective target input data array corresponding to the respective target date of the set of target dates, wherein, as a result of the processing by the trained first one or more layers, each of the plurality of embedding arrays that corresponds to the respective target date is dependent on the respective input data array for the target date and dates other than the target date in the set of target dates; and
transmitting the plurality of embedding arrays to a remote computing system over a network, for training a forecasting model.
10. The data processing system of claim 9 , further comprising a network interface,
wherein the data processing system is arranged to transmit the plurality of embedding arrays to the remote computing system via the network interface.
11. The data processing system of claim 9 , wherein the neural network is a deep neural network, and
the first one or more layers of the deep neural network include one or more recurrent neural network layers.
12. The data processing system of claim 11 , wherein the first one or more layers of the deep neural network includes at least one long short-term memory unit.
13. The data processing system of claim 9 , wherein the respective target date is a training date of the set of training dates.
14. The data processing system of claim 9 , wherein for each training date of the set of training dates, the target output value corresponds to an evaluation of a financial metric at the training date.
15. A non-transitory storage medium comprising machine readable instructions which, when executed by processing circuitry of a data processing system, cause the data processing system to execute a method including:
performing processing to generate a plurality of embedding arrays for a set of target dates, each of the plurality of embedding arrays having a first number of dimensions and representing information associated with a respective target date of the set of target dates, the performing the processing including, for each sequence of dates including training dates from a set of training dates:
receiving, for each training date of a respective sequence of dates, a respective input data array, wherein each respective input data array is received sequentially in a chronological order based on the training dates of the respective sequence of dates, each respective input data array having a second number of dimensions greater than the first number of dimensions and representing values of a predetermined set of date-dependent features;
receiving a target output value corresponding to an evaluation of a predetermined metric for each training date; and
performing an update routine including:
processing each respective input data array sequentially received for each training date in the respective sequence of dates using a first one or more layers of a neural network to generate an intermediate data array for a last training date in the respective sequence of dates, wherein the intermediate data array has the first number of dimensions;
processing the intermediate data array using second one or more layers of the neural network to generate a network output value,
determining an error between the network output value and the target output value for the last training date in the respective sequence of dates, and
updating values of a set of parameters of the neural network in a direction of a negative gradient of the determined error between the network output value and the target output value,
wherein, when the update routine has been performed for each training date of the set of training dates, the trained first one or more layers are obtained, and
wherein the method further includes:
generating the plurality of embedding arrays using the trained first one or more layers, by processing respective target input data arrays, each respective target input data array corresponding to the respective target date of the set of target dates, wherein, as a result of the processing by the trained first one or more layers, each of the plurality of embedding arrays that corresponds to the respective target date is dependent on the respective input data array for the target date and dates other than the target date in the set of target dates; and
transmitting the plurality of embedding arrays to a remote computing system over a network, for training a forecasting model.