Pre-computing data metrics using neural networks
Methods, system, and apparatus for pre-computing data metrics. A method can include receiving, by one or more computers, (i) data from a user including a plurality of data metrics, and (ii) historical data associated with the user; generating, by the one or more computers and based at least on the historical data associated with the user, a forecast for the user using an artificial neural network, the forecast indicating a prediction of future user behaviors; determining, by the one or more computers and based at least on the forecast for the user, a subset of data metrics, from among the plurality of data metrics, that are likely to be used by the user; performing, by the one or more computers, one or more automatic data operations on the subset of data metrics; and providing, by the one or more computers, the processed subset of data metrics for output.
1. A computer-implemented method comprising:
obtaining first data indicative of a set of historical actions performed by a set of users through an application;
generating, based on clustering the first data, second data indicative of a set of simulated historical actions performed by the set of users, wherein the second data reproduce statistical properties of the first data;
determining, for each historical action indicated by the first data, a first likelihood of use by a user at a later time point, wherein the user belongs to the set of users;
determining, for each simulated historical action indicated by the second data, a second likelihood of use by the user at a later time point;
training, based at least on the first and second likelihoods, a neural network model configured to determine a user forecast indicative of a set of future actions to be performed by the user, wherein the neural network comprises an input layer, a hidden layer, and an output layer;
determining, based on accessing the neural network model, a user forecast of a first user, wherein the user forecast is indicative of a set of future actions to be performed by the first user through the application at a later time point, and the user forecast includes a set of data metrics accessible through the application;
obtaining, for each data metric of the user forecast of the first user, an importance score, wherein the importance score is based on frequency of use of a respective data metric by the first user;
selecting, based on the importance score for each of the data metrics, a subset of data metrics from the set of data metrics, wherein the subset of data metrics only includes data metrics that are predicted to be used by the first user in a data processing operation at the later time point;
generating a metric table, wherein the metric table includes a set of quantitative information, wherein each quantitative information is associated with a corresponding one of the data metrics of the subset of data metrics, and wherein generating the metric table comprises:
transforming the first data and the second data into a star schema that includes one or more fact tables;
determining, based on applying a first data metric to the star schema, first quantitative information for the first data metric included in the subset of data metrics, wherein the first data metric is associated with a first future action included in the set of future actions;
determining, based on applying a second data metric to the star schema, second quantitative information for the second data metric included in the subset of data metrics, wherein the second data metric is associated with a second future action that is (i) included in the set of future actions and (ii) different from the first data metric;
determining, based on the first and the second quantitative information, aggregated quantitative information; and
storing the first quantitative information, the second quantitative information, and the aggregated quantitative information in the metric table; and
providing the metric table, wherein the metric table is accessible before the set of future actions is performed by the first user.
2. The method of claim 1 , wherein:
the input layer includes a plurality of input nodes each of which receives a subset of the data indicating historical actions performed by the user through the application, each subset representing a different feature of user behavior exhibited by the user when previously using the application;
the hidden layer includes a plurality of hidden nodes that each approximates the set of future actions based on a trained statistical model for the user; and
the output layer includes one or more output nodes that determine the set of future actions.
3. The method of claim 1 , comprising:
after determining the user forecast of the first user, receiving data indicating one or more products associated with the first user;
identifying, using the neural network model, a second subset of data metrics, from among the set of data metrics, that correspond to the one or more products associated with the first user;
determining that the user forecast of the first user has changed based on the second subset of data metrics; and
in response to determining that the user forecast of the first user has changed, adjusting the user forecast of the first user based at least on the data indicating one or more products.
4. The method of claim 1 , wherein the set of historical actions performed by the set of users comprises prior data metrics computed by the set of users through the application.
5. The method of claim 1 , wherein the set of users comprise one or more other users that are determined to be similar to the first user.
6. The method of claim 1 , wherein selecting the subset of data metrics comprises:
computing, for each data metric included in the set of data metrics, likelihood that a particular data metric will be used by the first user through the application at the later time point, wherein the likelihood is computed based on the set of historical actions; and
designating data metrics that have likelihoods greater than a threshold value as being included in the subset of data metrics.
7. The method of claim 1 , wherein the metric table is locally stored on a client device of the first user.
8. The method of claim 7 , wherein providing the metric table comprises configuring the client device to execute the application at the later time point such that the first quantitative information, the second quantitative information, and the aggregated quantitative information are displayed on a data analytics dashboard provided through the application.
9. A system comprising: one or more computers; and
a non-transitory computer-readable medium coupled to the one or more computers having instructions stored thereon, which, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
obtaining first data indicative of a set of historical actions performed by a set of users through an application;
generating, based on clustering the first data, second data indicative of a set of simulated historical actions performed by the set of users, wherein the second data reproduce statistical properties of the first data;
determining, for each historical action indicated by the first data, a first likelihood of use by a user at a later time point, wherein the user belongs to the set of users;
determining, for each simulated historical action indicated by the second data, a second likelihood of use by the user at a later time point;
training, based at least on the first and second likelihoods, a neural network model configured to determine a user forecast indicative of a set of future actions to be performed by the user, wherein the neural network comprises an input layer, a hidden layer, and an output layer;
determining, based on accessing the neural network model, a user forecast of a first user, wherein the user forecast is indicative of a set of future actions to be performed by the first user through the application at a later time point, and the user forecast includes a set of data metrics accessible through the application;
obtaining, for each data metric of the user forecast of the first user, an importance score, wherein the importance score is based on frequency of use of a respective data metric by the first user;
selecting, based on the importance score for each of the data metrics, a subset of data metrics from the set of data metrics, wherein the subset of data metrics only includes data metrics that are predicted to be used by the first user in a data processing operation at the later time point;
generating a metric table, wherein the metric table includes a set of quantitative information, wherein each quantitative information is associated with a corresponding one of the data metrics of the subset of data metrics, and wherein generating the metric table comprises:
transforming the first data and the second data into a star schema that includes one or more fact tables;
determining, based on applying a first data metric to the star schema, first quantitative information for the first data metric included in the subset of data metrics, wherein the first data metric is associated with a first future action included in the set of future actions;
determining, based on applying a second data metric to the star schema, second quantitative information for the second data metric included in the subset of data metrics, wherein the second data metric is associated with a second future action that is (i) included in the set of future actions and (ii) different from the first data metric;
determining, based on the first and the second quantitative information, aggregated quantitative information; and
storing the first quantitative information, the second quantitative information, and the aggregated quantitative information in the metric table; and
providing the metric table, wherein the metric table is accessible before the set of future actions is performed by the first user.
10. The system of claim 9 , wherein:
the input layer includes a plurality of input nodes each of which receives a subset of the data indicating historical actions performed by the user through the application, each subset representing a different feature of user behavior exhibited by the user when previously using the application;
the hidden layer includes a plurality of hidden nodes that each approximates the set of future actions based on a trained statistical model for the user; and
the output layer includes one or more output nodes that determine the set of future actions.
11. The system of claim 9 , wherein the operations comprise:
after determining the user forecast of the first user, receiving, by the one or more computers, data indicating one or more products associated with the first user;
identifying, by the one or more computers and using the neural network model, a second subset of data metrics, from among the set of data metrics, that correspond to the one or more products associated with the first user;
determining, by the one or more computers, that the user forecast of the first user has changed based on the second subset of data metrics; and
in response to determining that the user forecast of the first user has changed, adjusting, by the one or more computers, the user forecast of the first user based at least on the data indicating one or more products.
12. The system of claim 9 , wherein the set of historical actions performed by the set of users comprises prior data metrics computed by the set of users through the application.
13. A non-transitory computer storage device encoded with a computer program, the computer program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining first data indicative of a set of historical actions performed by a set of users through an application;
generating, based on clustering the first data, second data indicative of a set of simulated historical actions performed by the set of users, wherein the second data reproduce statistical properties of the first data;
determining, for each historical action indicated by the first data, a first likelihood of use by a user at a later time point, wherein the user belongs to the set of users;
determining, for each simulated historical action indicated by the second data, a second likelihood of use by the user at a later time point;
training, based at least on the first and second likelihoods, a neural network model configured to determine a user forecast indicative of a set of future actions to be performed by the user, wherein the neural network comprises an input layer, a hidden layer, and an output layer;
determining, based on accessing the neural network model, a user forecast of a first user, wherein the user forecast is indicative of a set of future actions to be performed by the first user through the application at a later time point, and the user forecast includes a set of data metrics accessible through the application;
obtaining, for each data metric of the user forecast of the first user, an importance score, wherein the importance score is based on frequency of use of a respective data metric by the first user;
selecting, based on the importance score for each of the data metrics, a subset of data metrics from the set of data metrics, wherein the subset of data metrics only includes data metrics that are predicted to be used by the first user in a data processing operation at the later time point;
generating a metric table, wherein the metric table includes a set of quantitative information, wherein each quantitative information is associated with a corresponding one of the data metrics of the subset of data metrics, and wherein generating the metric table comprises:
transforming the first data and the second data into a star schema that includes one or more fact tables;
determining, based on applying a first data metric to the star schema, first quantitative information for the first data metric included in the subset of data metrics, wherein the first data metric is associated with a first future action included in the set of future actions;
determining, based on applying a second data metric to the star schema, second quantitative information for the second data metric included in the subset of data metrics, wherein the second data metric is associated with a second future action that is (i) included in the set of future actions and (ii) different from the first data metric;
determining, based on the first and the second quantitative information, aggregated quantitative information; and
storing the first quantitative information, the second quantitative information, and the aggregated quantitative information in the metric table; and
providing the metric table, wherein the metric table is accessible before the set of future actions is performed by the first user.
14. The non-transitory computer storage device of claim 13 , wherein:
the input layer includes a plurality of input nodes each of which receives a subset of the data indicating historical actions performed by the user through the application, each subset representing a different feature of user behavior exhibited by the user when previously using the application;
the hidden layer includes a plurality of hidden nodes that each approximates the set of future actions based on a trained statistical model for the user; and
the output layer includes one or more output nodes that determine the set of future actions.
15. The non-transitory computer storage device of claim 13 , wherein the operations comprise:
after determining the user forecast of the first user, receiving, by the one or more computers, data indicating one or more products associated with the first user;
identifying, by the one or more computers and using the neural network model, a second subset of data metrics, from among the set of data metrics, that correspond to the one or more products associated with the first user;
determining, by the one or more computers, that the user forecast of the first user has changed based on the second subset of data metrics; and
in response to determining that the user forecast of the first user has changed, adjusting, by the one or more computers, the user forecast of the first user based at least on the data indicating one or more products.