System and methods for finding hidden topics of documents and preference ranking documents
View Patent ↗Systems and methods are disclosed to perform preference learning on a set of documents includes receiving raw input features from the set of documents stored on a data storage device; generating polynomial combinations from the raw input features; generating one or more parameters; applying the parameters to one or more classifiers to generate outputs; determining a loss function and parameter gradients and updating parameters determining one or more sparse regularizing terms and updating the parameters; and expressing that one document is preferred over another in a search query and retrieving one or more documents responsive to the search query.
1. A computer-implemented method to apply a query to a set of documents, comprising:
by a computer:
reconstructing a document term matrix XεR N×M where N is a number of documents and M is a number of words, by minimizing reconstruction errors with min ∥X−UA∥, where A is a fixed projection matrix and U is a column orthogonal matrix;
determining a loss function and parameter gradients to generate U;
fixing U while determining the loss function and sparse regularization constraints on the projection matrix A;
generating parameter coefficients and generating a sparse projection matrix A; and
generating a Sparse Latent Semantic Analysis (Sparse SLA) model and applying the model to a set of documents and displaying documents matching a query.
2. The computer-implemented method of claim 1 , wherein the document-term matrix comprises XεR N×M , where N is the number of documents and M is the number of words, and where a column orthogonal matrices UεR N×D (U T U=I) and VεR M×D (V T V=I) represent document and word embeddings into a latent space, and where I comprises a unity matrix.
3. The method of claim 1 , comprising automatically selecting most relevant words for each latent topic.
4. The method of claim 1 , comprising projecting documents with the sparse projection matrix.
5. The method of claim 1 , comprising adding an entry-wise l 1 -norm of A as a regularization term to the loss function and formulating the Sparse LSA model as:
min
U
,
A
1
2
X
-
UA
F
2
+
λ
A
1
subject
-
to
:
U
T
U
=
I
,
where
A
1
=
∑
d
=
1
D
∑
j
=
1
M
a
dj
is the entry-wise l 1 -norm of A and λ is a regularization parameter for a density (the number of nonzero entries) of A.
6. The method of claim 1 , comprising optimizing A according to the following relationship:
min
A
1
2
X
-
UA
+
λ
A
1
.
7. The method of claim 6 , comprising solving M independent optimizations according to the following relationship:
min
A
j
1
2
X
j
-
UA
j
2
2
+
λ
A
j
1
;
j
=
1
,
…
,
M
.
8. The method of claim 1 , comprising initializing
U
0
=
(
I
D
0
)
,
where I comprises a unity matrix and D comprises a dimensionality of a latent space.
9. The method of claim 1 , comprising iterating until convergence of U and A .
10. The method of claim 1 , comprising determining A by solving M LASSO problems.
11. The method of claim 10 , comprising determining:
min
A
j
1
2
X
j
-
UA
j
2
2
+
λ
A
j
1
;
j
=
1
,
…
,
M
.
12. The method of claim 1 , comprising projecting X onto a latent space V=XA T .
13. The method of claim 1 , comprising generating V:V=PΔQ where U=PQ.
14. The method of claim 1 , comprising:
determining A by solving M LASSO problems;
projecting X onto a latent space V=XA T ;
determining V=PΔQ where U=PQ; and
generating the sparse projection matrix A.
15. The method of claim 1 , comprising determining a Group Structured Sparse LSA.
16. The method of claim 15 , comprising using a group structured sparsity-inducing penalty to select most relevant groups of features relevant to a latent topic.
17. The method of claim 1 , comprising enforcing a non-negative constraint on the projection matrix.
18. The method of claim 17 , comprising determining
min
U
,
A
1
2
X
-
UA
F
2
+
λ
A
1
subject
to
U
T
U
=
I
,
A
≥
0.
19. The method of claim 17 , comprising determining
min
U
,
A
1
2
X
-
UA
F
2
+
λ
∑
d
=
1
D
∑
j
=
1
J
a
dj
subject
to
U
T
U
=
I
,
A
≥
0.
20. A system to perform preference learning on a set of documents, comprising a computing device including a processor and a memory coupled to said processor, said memory having stored thereon computer executable instructions that upon execution by the processor cause the system to:
receive raw input features from the set of documents stored on a data storage device and generate polynomial combinations from the raw input features;
generate one or more parameters;
apply the parameters to one or more classifiers to generate outputs;
determine a loss function and parameter gradients and updating parameters determining one or more sparse regularizing terms and updating the parameters; and
expressing that one document is preferred over another in a search query and retrieving one or more documents responsive to the search query.