Method and system for query training
The method for query training can include: determining a graphical representation, determining an inference network based on the graphical representation, determining a query distribution, sampling one or more train queries from the query distribution, and optionally determining a trained inference network by training the untrained inference network using the train query. The method can optionally include determining an inference query and determining an inference query result for the inference query using the trained inference network.
1. A method, comprising:
determining a graphical representation comprising graphical parameters;
determining inference equations for the graphical representation, wherein the inference equations comprise the graphical parameters;
determining an untrained inference neural network, comprising the graphical parameters, by unrolling the inference equations for a predetermined number of iterations into the untrained inference neural network;
determining a query distribution;
sampling a plurality of train queries from the query distribution;
determining a trained inference neural network by training the untrained inference neural network to determine estimated output values of output variables of a unary representation, wherein the output variables are determined with a train query of the plurality of train queries; and
determining an inference query result for an inference query using the trained inference neural network, wherein the inference query is within the query distribution but different from the train queries.
2. The method of claim 1 , wherein the graphical representation is an undirected probabilistic graphical model.
3. The method of claim 1 , wherein the graphical representation comprises a hidden variable.
4. The method of claim 1 , wherein the trained inference neural network comprises values for the graphical parameters, determined by training the untrained inference neural network.
5. The method of claim 4 , wherein the values for the graphical parameters are the same for each layer of the trained inference neural network.
6. The method of claim 1 , wherein the train queries and the inference query are binary valued vectors.
7. The method of claim 1 , wherein training the untrained inference neural network comprises:
receiving the training data, comprising input variables, the output variables, and associated ground truth values for the input variables and the output variables, from a training dataset;
determining the unary representation comprising the input variables and the output variables by masking out the ground truth values of the output variables using the train query of the plurality of train queries;
feeding the unary representation into the untrained inference network, wherein the inference neural network determines estimated values for each variable of the unary representation;
comparing the estimated output values of the output variables and the ground truth values of the output variables; and
updating the graphical parameters of the untrained inference neural network based on the comparison.
8. The method of claim 7 , wherein each value of the estimated output variables is estimated by conditioning on all of the ground truth values of the input variables.
9. The method of claim 7 , wherein the input variables and the output variables are continuous.
10. The method of claim 9 , wherein each variable is approximated by a distribution with known statistics.
11. The method of claim 10 , wherein the distribution is a gaussian distribution.
12. The method of claim 7 , wherein the variables represent an image, wherein the input variables represent known image pixel values and the output variables represent image pixels that are estimated by the untrained inference network.
13. The method of claim 1 , wherein the inference equations are determined based on a potential function of the graphical representation, wherein the potential function comprises the graphical parameters.
14. The method of claim 1 , wherein the inference equations are derived from belief propagation equations.
15. The method of claim 1 , the graphical representation is a Markov random field.
16. The method of claim 1 , the query distribution is determined based on a use case.
17. The method of claim 16 , the untrained inference neural network is further trained on training data, wherein the training data is sampled from a data distribution determined based on the use case.
18. A system, comprising a non-transitory computer readable medium having stored thereon software instructions that, when executed by a processing system, cause the processing system to estimate inference variable values for inference variables of an inference query using a trained inference neural network, wherein the trained inference neural network is determined by:
determining a query distribution;
sampling a train query from the query distribution, wherein the train query is different from the inference query;
receiving variables and associated values from a training dataset;
splitting the variables into input variables and output variables using the train query;
outputting estimated values for the output variables and values for the input variables by an untrained inference neural network, wherein the untrained inference neural network comprises N layers and is defined by unrolling inference equations, derived from a graphical representation, for N iterations;
training the inference neural network based on a comparison of the estimated values and known values for the output variables.
19. The method of claim 18 , wherein the inference variable values are associated with an inference query, and wherein the train queries and the inference query are both within the query distribution.
20. The method of claim 18 , wherein the N layers all share a common set of weights, wherein the weights are determined from graphical parameters of the graphical representation.