Systems and methods for modeling probability distributions

1. A method for generating a Deep Boltzmann Machine (DBM), wherein the method comprises:

generating values from a first restricted Boltzmann machine (RBM) comprising a visible layer and a hidden layer using a Markov chain Monte Carlo (MCMC) method by performing at least one iteration comprising:

generating, from a first set of visible values from the visible layer, a set of hidden values in the hidden layer of the first RBM; and

generating a second set of visible values in the visible layer of the first RBM based on the generated set of hidden values;

computing a set of likelihood gradients based on at least one of the first set of visible values and the generated second set of visible values;

computing, based on at least one of the set of hidden values and the second set of visible values, a set of adversarial gradients using an adversarial model, wherein the adversarial model includes an architecture and weights that are the same as that of the first RBM;

computing a set of compound gradients based on the set of likelihood gradients and the set of adversarial gradients;

updating the first RBM based on the set of compound gradients;

sampling the hidden layer from the first RBM;

stacking the visible layer and the hidden layer from the first RBM into a vector;

training a second RBM having a visible layer and a hidden layer, wherein the vector is the visible layer of the second RBM and the hidden layer of the second RBM is equivalent to a layer between the visible layer and the hidden layer of the first RBM; and

generating the DBM by copying weights from the first and second RBMs to the DBM, wherein the DBM comprises a visible layer and a plurality of hidden layers.

2. The method of claim 1 , wherein the visible layer of the first RBM comprises a composite layer composed of a plurality of sub-layers for different data types.

3. The method of claim 2 , wherein the plurality of sub-layers comprises at least one of a Bernoulli layer, an Ising layer, a one-hot layer, a von Mises-Fisher layer, a Gaussian layer, a ReLU layer, a clipped ReLU layer, a student-t layer, an ordinal layer, an exponential layer, and a composite layer.

4. The method of claim 1 further comprising:

receiving a phenotype vector for a patient;

using the first RBM to generate a time progression of a disease; and

treating the patient based on the generated time progression.

5. The method of claim 1 , wherein the visible layer of the first RBM and the hidden layer of the first RBM are for a first time instance, wherein the hidden layer of the first RBM is further connected to another hidden layer in the DBM that incorporates data from a different second time instance.

6. The method of claim 1 , wherein the visible layer of the first RBM is a composite layer comprising data for a plurality of different time instances.

7. The method of claim 1 , wherein computing the set of likelihood gradients comprises performing Gibbs sampling.

8. The method of claim 1 , wherein the set of compound gradients are weighted averages of the set of likelihood gradients and the set of adversarial gradients.

9. The method of claim 1 further comprising training the adversarial model by:

drawing data samples based on authentic data;

drawing fantasy samples based on values from the first RBM; and

training the adversarial model based on the adversarial model's ability to distinguish between the data samples and the fantasy samples.

10. The method of claim 9 , wherein training the adversarial model comprises measuring a probability that a particular sample is drawn from either the authentic data or the first RBM.

11. The method of claim 1 , wherein the adversarial model is one of a fully-connected classifier, a logistic regression model, a nearest neighbor classifier, and a random forest.

12. The method of claim 1 further comprising using the first RBM to generate a set of samples of a target population.

13. The method of claim 1 , wherein computing a set of likelihood gradients comprises computing a convex combination of a Monte Carlo estimate and a mean field estimate.

14. The method of claim 1 , wherein computing a set of likelihood gradients comprises:

initializing a plurality of samples;

initializing an inverse temperature for each sample of the plurality of samples; and

for each sample of the plurality of samples:

updating the inverse temperature by sampling from an autocorrelated Gamma distribution; and

updating the sample using Gibbs sampling.

15. A non-transitory machine readable medium containing processor instructions for generating a Deep Boltzmann Machine (DBM), wherein execution of the instructions by a processor causes the processor to perform a process that comprises:

generating values from a first restricted Boltzmann machine (RBM) comprising a visible layer and a hidden layer using a Markov chain Monte Carlo (MCMC) method by performing at least one iteration comprising:

generating, from a first set of visible values from the visible layer, a set of hidden values in the hidden layer of the first RBM; and

generating a second set of visible values in the visible layer of the first RBM based on the generated set of hidden values;

computing a set of likelihood gradients based on at least one of the first set of visible values and the generated second set of visible values;

computing, based on at least one of the set of hidden values and the second set of visible values, a set of adversarial gradients using an adversarial model, wherein the adversarial model includes an architecture and weights that are the same as that of the first RBM;

computing a set of compound gradients based on the set of likelihood gradients and the set of adversarial gradients;

updating the first RBM based on the set of compound gradients;

sampling the hidden layer from the first RBM;

stacking the visible layer and the hidden layer from the first RBM into a vector;

training a second RBM having a visible layer and a hidden layer, wherein the vector is the visible layer of the second RBM and the hidden layer of the second RBM is equivalent to a layer between the visible layer and the hidden layer of the first RBM; and

generating the DBM by copying weights from the first and second RBMs to the DBM, wherein the DBM comprises a visible layer and a plurality of hidden layers.

16. The non-transitory machine readable medium of claim 15 , wherein the visible layer of the first RBM comprises a composite layer composed of a plurality of sub-layers for different data types.