System and method for generative model for stochastic point processes
A variational auto-encoder model is trained to generate probabilities of action categories and probabilities of inter-arrival times of next action from a sequence of past actions by generating a concatenated representation of each action and associated time, encoding the concatenated representations, determining a conditional prior distribution for a next action, determining a conditional posterior distribution for the current action, sampling a latent variable from the conditional prior distribution, generating a probability distribution over a current action category, and generating a probability distribution over inter-arrival times for the current action category.
1 . A computer-implemented method for training a variational auto-encoder model for asynchronous future action prediction, comprising:
receiving data representing events, the events being discrete events occurring in continuous time, the events including past events and a current event, the events including irregularly time spaced events that do not occur at regular time intervals, wherein the model does not have any information about the current event, wherein the current event does not have a discernable causal relationship with the past events;
for each of the events, generating an event representation, the event representation based on an event category selected from a plurality of event categories;
generating a time representation of inter-arrival times between each of the events;
generating a concatenated representation of each event representation and associated time representation;
encoding the concatenated representations of the past events into a past vector representation;
encoding the concatenated representations of the current event into a current vector representation;
determining a conditional prior distribution for a next event, based at least in part on the past vector representation;
determining a conditional posterior distribution for the current event, based at least in part on the current vector representation;
sampling a latent variable from the conditional prior distribution;
generating a probability distribution over the plurality of event categories for a current event category using an action decoder neural network;
generating a probability distribution over inter-arrival times for the current event category using a time decoder neural network, wherein the probability distribution over inter-arrival times is parametrized by the time decoder neural network, wherein the time decoder neural network accepts the sampled latent variable as input, wherein the time decoder neural network includes at least one time decoder neural network activation layer and at least one time decoder neural network layer that applies an exponential non-linear mapping to an output of the at least one time decoder neural network activation layer to generate a positive parameter for the probability distribution over inter-arrival times;
updating a parameter of the conditional prior distribution using the action decoder neural network and the time decoder neural network such that the parameter of the conditional prior distribution shifts over training to improve accuracy of labelling a predicted next event with a label representing an event category of the plurality of event categories;
generating one or more labels for a predicted next event in a sequence of events or for performing a next event based on the generated probability distributions over the current event category and the inter-arrival times, wherein the one or more labels for the predicted next event is based on a most probable event category from the probability distribution over the plurality of event categories for a current event category, wherein the predicted next event has a predicted inter-arrival time based on the probability distribution over inter-arrival times and the positive parameter for the probability distribution over inter-arrival times; and
communicating via a user interface the one or more labels for the predicted next event.
2 . The computer-implemented method of claim 1 , further comprising:
optimizing a variational lower bound comprising a difference between a log likelihood of ground truth data and a Kullback-Leibler divergence between the true and conditional posterior distributions.
3 . The computer-implemented method of claim 1 , further comprising modelling the probability distribution over inter-arrival times for the current event category with a non-parametric temporal point process distribution using a normalizing flow framework.
4 . The computer-implemented method of claim 1 , wherein the conditional prior distribution is time-variant and a function of all past events except the current event.
5 . The computer-implemented method of claim 1 , wherein the conditional prior distribution or the conditional posterior distribution is a Gaussian distribution.
6 . The computer-implemented method of claim 1 , wherein the conditional prior distribution and the conditional posterior distribution are modelled as a multivariate Gaussian distributions with diagonal covariance.
7 . The computer-implemented method of claim 1 , further comprising modelling the probability distribution over the current event category with a multinomial distribution.
8 . The computer-implemented method of claim 1 , further comprising modelling the probability distribution over inter-arrival times for the current event category with a parameterized exponential distribution.
9 . The computer-implemented method of claim 3 , wherein the temporal point process distribution follows an exponential distribution.
10 . The computer-implemented method of claim 1 , wherein the encoding the concatenated representations of the past events into a past vector representation is with a long short-term memory network.
11 . The computer-implemented method of claim 1 , wherein the encoding the concatenated representations of the current event into a current vector representation is with a long short-term memory network.
12 . The computer-implemented method of claim 1 , wherein the events are irregularly spaced in time.
13 . A non-transitory computer readable medium comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer cause the computer to:
predict probabilities of event categories and probabilities of inter-arrival times of a next event, using a model trained using the method of claim 1 .
14 . A computer-implemented method for generating probabilities of event categories and probabilities of inter-arrival times of a next event from a sequence of past events using a variational auto-encoder model for asynchronous future action prediction, comprising:
receiving data representing the past events, the past events and the next event being discrete events occurring in continuous time, the events including irregularly time spaced events that do not occur at regular time intervals, wherein the model does not have any information about the next event, wherein the next event does not have a discernable causal relationship with the past events;
generating an event representation, the event representation based on an event category selected from a plurality of event categories for each of the past events;
generating a time representation of inter-arrival times between each of the past events;
generating a concatenated representation of each event representation and associated time representation;
encoding the concatenated representations into a vector representation;
determining a conditional posterior distribution for the next event;
sampling a latent variable from the conditional posterior distribution;
generating a probability distribution over the plurality of events categories for a next event category using an action decoder neural network;
generating a probability distribution over inter-arrival times for the next event category using a time decoder neural network, wherein the probability distribution over inter-arrival times is parametrized by the time decoder neural network, wherein the time decoder neural network accepts the sampled latent variable as input, wherein the time decoder neural network includes at least one time decoder neural network activation layer and at least one time decoder neural network layer that applies an exponential non-linear mapping to an output of the at least one time decoder neural network activation layer to generate a positive parameter for the probability distribution over inter-arrival times; updating a parameter of the conditional prior distribution using the action decoder neural network and the time decoder neural network such that the parameter of the conditional prior distribution shifts over training to improve accuracy of labelling a predicted next event with a label representing an event category of the plurality of event categories;
generating one or more labels for a predicted next event in a sequence of events or for performing a next event based on the generated probability distributions over the current event category and the inter-arrival times, wherein the one or more labels for the predicted next event is based on a most probable event category from the probability distribution over the plurality of event categories for a current event category, wherein the predicted next event has a predicted inter-arrival time based on the probability distribution over inter-arrival times and the positive parameter for the probability distribution over inter-arrival times; and
communicating via a user interface the one or more labels for the predicted next event.
15 . The computer-implemented method of claim 14 , wherein the encoding the concatenated representations into a current vector representation is with a long short-term memory network.
16 . The computer-implemented method of claim 14 , further comprising modelling the probability distribution over the next event category with a multinomial distribution.
17 . The computer-implemented method of claim 14 , further comprising modelling the probability distribution over inter-arrival times for the next event category with a parameterized exponential distribution.
18 . The computer-implemented method of claim 14 , further comprising modelling the probability distribution over inter-arrival times for the next event category with a non-parametric temporal point process distribution using a normalizing flow framework.
19 . The computer-implemented method of claim 14 , wherein the past events are irregularly spaced in time.
20 . The computer-implemented method of claim 1 wherein the data representing the events is video data.
21 . The computer-implemented method of claim 1 wherein the generated one or more outputs comprises a latent representation in the form of a vector representing a current state of events, wherein the latent representation can be used for matching.