Processing high-dimensional data via EM-style iterative algorithm
View Patent ↗Systems and methods are disclosed for factorizing high-dimensional data by simultaneously capturing factors for all data dimensions and their correlations in a factor model, wherein the factor model provides a parsimonious description of the data; and generating a corresponding loss function to evaluate the factor model.
1. A method for factorizing high-dimensional data, comprising:
simultaneously capturing factors for all data dimensions and their correlations in a factor model; and
generating a corresponding loss function to evaluate the factor model expressed as a non-negative tensor factorization and to minimize a predetermined loss between observed data and data predicted by the generative model; and
applying EM-style iterative algorithm for solving the non-negative tensor factorization.
2. The method of claim 1 , comprising generating a data tensor from the high dimensional input data.
3. The method of claim 1 , comprising generating one or more factors and factor relationships using a factorization algorithm.
4. The method of claim 1 , comprising applying the factors and their relationships to a recommendation application.
5. The method of claim 1 , comprising applying the factors and their relations to the clustering application.
6. The method of claim 1 , comprising applying the factors and their relations to the summarization application.
7. The method of claim 1 , comprising applying a Kullback-Leibler divergence to measure an error in tensor factorization.
8. The method of claim 1 , comprising applying alternative algorithm to learn the model parameters.
9. The method of claim 1 , comprising applying a Frobenius tensor norm to measure an error in tensor factorization.
10. The method of claim 1 , comprising deriving a probabilistic factor model.
11. The method of claim 10 , wherein the probabilistic factor model is used generate a discriminative version and a version containing regularization on result sparseness.
12. The method of claim 1 , comprising approximating the data using a non-negative tensor factorization.
13. The method of claim 1 , comprising generating a set of matrices which represent the factors in different dimensions of the data.
14. The method of claim 1 , comprising generating a tensor representing correlations among the factors in different dimensions of data.
15. A system for factorizing high-dimensional observed data, comprising:
means for deriving a generative model that simultaneously captures factors for all data dimensions and their correlations;
means for generating predicted data and mapping the generative model into a non-negative tensor factorization and means for minimizing a predetermined loss between observed data and data predicted by the generative model; and
means for minimizing the predetermined loss between the observed data and the predicted data and to generate a tensor that represents the correlations among the factors in different dimensions of data by applying EM-style iterative algorithm for solving the non-negative tensor factorization.
16. The system of claim 15 , comprising means for generating a set of matrices which represent the factors in different dimensions of the data with the regenerative model including Cx 1 X 1 x 2 X 2 . . . x N X N , where X 1 , . . . , X N are the factors in first, second, . . . , and N-th dimensions, C is a core tensor to capture a correlation among the factors X 1 , . . . , X N , where X 1 , . . . , X N are nonnegative and a sum of each column of X 1 , . . . , X N is one.
17. The system of claim 15 , comprising means for generating a tensor representing correlations among the factors in different dimensions of data.
18. The system of claim 15 , comprising means for applying EM-style iterative algorithm for solving the non-negative tensor factorization (NTF).