Directed Acyclic Graph of Recommendation Dimensions
A computer accesses a dataset for computing an action including multiple decision items and a configuration for a directed acyclic graph (DAG) comprising nodes representing dimensions. Each dimension is associated with a decision item of the action. The computer computes a value for a first decision item associated with a top level dimension of the DAG based on the dataset and using a reinforcement learning agent for the top level dimension. The computer computes a value for a second decision item associated with a non-top level dimension from the DAG based on the dataset and a specified subset of outputs of reinforcement learning engines for dimensions upstream from the non-top level dimension in the DAG and using a reinforcement learning agent for the non-top level dimension. The computer provides the computed action including the value for the first decision item and the value for the second decision item.
1 . A method comprising:
accessing, at a computing machine, an input dataset for computing an action including multiple decision items and a configuration for a directed acyclic graph comprising nodes representing dimensions, each dimension being associated with a decision item of the action;
computing a value for a first decision item associated with a top level dimension of the directed acyclic graph, wherein the value for the first decision item is computed based on the input dataset and using a reinforcement learning agent for the top level dimension;
computing a value for a second decision item associated with a non-top level dimension from the directed acyclic graph, wherein the value for the second decision item is computed based on the input dataset and a specified subset of outputs of reinforcement learning engines for dimensions upstream from the non-top level dimension in the directed acyclic graph and using a reinforcement learning agent for the non-top level dimension; and
providing, via the computing machine, the computed action including at least the value for the first decision item and the value for the second decision item.
2 . The method of claim 1 , wherein the directed acyclic graph comprises unidirectional edges between the multiple nodes, wherein an upstream dimension being upstream from the non-top level dimension comprises an edge existing from a node of the upstream dimension to a node of the non-top level dimension.
3 . The method of claim 1 , further comprising:
generating an ordered list of the dimensions of the directed acyclic graph based on a structure of the directed acyclic graph, wherein the top level dimension occupies a first position in the ordered list, wherein the dimensions upstream from the non-top level dimension appear prior to the non-top level dimension in the ordered list.
4 . The method of claim 1 , wherein the configuration for the direct acyclic graph is accessed via one or more configuration files.
5 . The method of claim 1 , wherein the configuration for the direct acyclic graph is generated via a graphical user interface.
6 . The method of claim 1 , wherein the top level dimension lacks upstream dimensions in the direct acyclic graph.
7 . The method of claim 1 , wherein the top level dimension is upstream from the non-top level dimension in the direct acyclic graph, wherein the specified subset of outputs of reinforcement learning agents associated with the dimensions upstream from the non-top level dimension comprises an output of the reinforcement learning agent for the top level dimension.
8 . The method of claim 1 , further comprising:
causing, via the computing machine, performance of the computed action;
determining a result of the computed action; and
iteratively training the reinforcement learning agent for the top level dimension or the reinforcement learning agent for the non-top level dimension based on the determined result.
9 . The method of claim 1 , further comprising:
computing a value for at least one decision item associated with a Nth dimension of the directed acyclic graph, wherein N is a positive integer corresponding to a position of the Nth dimension in a linearization of the directed acyclic graph comprising at least N dimensions, wherein the value of the decision item is computed based on the input dataset and a specified subset of outputs of reinforcement learning engines for dimensions upstream from the Nth dimension in the directed acyclic graph and using a reinforcement learning agent for the Nth dimension.
10 . The method of claim 1 , wherein the action comprises transmitting an offer message to a user, wherein the multiple decision items comprise a channel, a day, a time, and an offer message identifier.
11 . A non-transitory machine-readable medium storing instructions which, when executed by one or more computing machines, cause the one or more computing machines to perform operations comprising:
accessing an input dataset for computing an action including multiple decision items and a configuration for a directed acyclic graph comprising nodes representing dimensions, each dimension being associated with a decision item of the action;
computing a value for a first decision item associated with a top level dimension of the directed acyclic graph, wherein the value for the first decision item is computed based on the input dataset and using a reinforcement learning agent for the top level dimension;
computing a value for a second decision item associated with a non-top level dimension from the directed acyclic graph, wherein the value for the second decision item is computed based on the input dataset and a specified subset of outputs of reinforcement learning engines for dimensions upstream from the non-top level dimension in the directed acyclic graph and using a reinforcement learning agent for the non-top level dimension; and
providing the computed action including at least the value for the first decision item and the value for the second decision item.
12 . The machine-readable medium of claim 11 , wherein the directed acyclic graph comprises unidirectional edges between the multiple nodes, wherein an upstream dimension being upstream from the non-top level dimension comprises an edge existing from a node of the upstream dimension to a node of the non-top level dimension.
13 . The machine-readable medium of claim 11 , wherein the configuration for the direct acyclic graph is accessed via one or more configuration files.
14 . The machine-readable medium of claim 11 , the operations further comprising:
generating an ordered list of the dimensions of the directed acyclic graph based on a structure of the directed acyclic graph, wherein the top level dimension occupies a first position in the ordered list, wherein the dimensions upstream from the non-top level dimension appear prior to the non-top level dimension in the ordered list.
15 . The machine-readable medium of claim 11 , wherein the configuration for the direct acyclic graph is generated via a graphical user interface.
16 . The machine-readable medium of claim 11 , wherein the top level dimension lacks upstream dimensions in the direct acyclic graph.
17 . A system comprising:
processing circuitry; and
a memory storing instructions which, when executed by the processing circuitry, cause the processing circuitry to perform operations comprising:
accessing an input dataset for computing an action including multiple decision items and a configuration for a directed acyclic graph comprising nodes representing dimensions, each dimension being associated with a decision item of the action;
computing a value for a first decision item associated with a top level dimension of the directed acyclic graph, wherein the value for the first decision item is computed based on the input dataset and using a reinforcement learning agent for the top level dimension;
computing a value for a second decision item associated with a non-top level dimension from the directed acyclic graph, wherein the value for the second decision item is computed based on the input dataset and a specified subset of outputs of reinforcement learning engines for dimensions upstream from the non-top level dimension in the directed acyclic graph and using a reinforcement learning agent for the non-top level dimension; and
providing the computed action including at least the value for the first decision item and the value for the second decision item.
18 . The system of claim 17 , wherein the directed acyclic graph comprises unidirectional edges between the multiple nodes, wherein an upstream dimension being upstream from the non-top level dimension comprises an edge existing from a node of the upstream dimension to a node of the non-top level dimension.
19 . The system of claim 17 , the operations further comprising:
generating an ordered list of the dimensions of the directed acyclic graph based on a structure of the directed acyclic graph, wherein the top level dimension occupies a first position in the ordered list, wherein the dimensions upstream from the non-top level dimension appear prior to the non-top level dimension in the ordered list.
20 . The system of claim 17 , wherein the configuration for the direct acyclic graph is accessed via one or more configuration files.