System and methods for finding hidden topics of documents and preference ranking documents
View Patent ↗Systems and methods are disclosed to perform preference learning on a set of documents includes receiving raw input features from the set of documents stored on a data storage device; generating polynomial combinations from the raw input features; generating one or more parameters; applying the parameters to one or more classifiers to generate outputs; determining a loss function and parameter gradients and updating parameters determining one or more sparse regularizing terms and updating the parameters; and expressing that one document is preferred over another in a search query and retrieving one or more documents responsive to the search query.
1. A method to perform preference learning on a set of documents, comprising:
receiving raw input features from the set of documents stored on a data storage device;
generating polynomial combinations from the raw input features;
generating one or more parameters W;
applying W to one or more classifiers to generate outputs;
determining a loss function and parameter gradients and updating W;
determining one or more sparse regularizing terms and updating W; and
expressing that one document is preferred over another in a search query and retrieving one or more documents responsive to the search query,
wherein said regularizing terms are responsive to an imposing of an entry-wise l 1 regularization on W and a refitting without the regularization is used to improve preference prediction while keeping the learned sparsity and said refitting reducing additional bias introduced by the l 1 regularization, said refitting being affected by
P
Ω
(
W
)
ij
=
{
W
ij
if
(
i
,
j
)
∈
Ω
0
if
(
i
,
j
)
not
∈
Ω
where W represents a relationship between a pair of words, Ω represents indices of non-zero entries of sparse W, and ij are matrix iterations.
2. The method of claim 1 , comprising constraining W to be a sparse matrix zero entry for pairs of words irrelevant to the preference learning.
3. The method of claim 1 , comprising enforcing an l 1 regularization with mini-batch shrinking for every predetermined iterations in a stochastic gradient descent.
4. The method of claim 1 , comprising performing a stochastic (sub)gradient descent (SGD) with an online learning framework.
5. The method of claim 1 , comprising applying a learning rate η t , a decaying learning rate is used with η 1 =C/√t, where C is a pre-defined constant as an initial learning rate and t represents an iteration .
6. The method of claim 1 , comprising shrinking W ij t with an absolute value less than λη t to zero and generating a sparse W matrix, λ is a regularization parameter which controls a sparsity level (number of nonzero entries) of W and η t denotes a learning rate.
7. The method of claim 1 , comprising performing shrinkage at every iteration to generate a sparse W .
8. A computer to perform preference learning on a set of documents, comprising:
means for receiving raw input features from the set of documents stored on a data storage device;
means for generating polynomial combinations from the raw input features ;
means for generating one or more parameters W;
means for applying W to one or more classifiers to generate outputs;
means for determining a loss function and parameter gradients and updating W;
means for determining one or more sparse regularizing terms and updating W; and
means for expressing that one document is preferred over another in a search query and retrieving one or more documents responsive to the search query,
wherein said regularizing terms are responsive to an imposing of an entry-wise l 1 regularization on W and to reduce additional bias introduced by the l 1 regularization a refitting without the regularization is used to improve preference prediction while keeping the learned sparsity, said refitting being related to
P
Ω
(
W
)
ij
=
{
W
ij
if
(
i
,
j
)
∈
Ω
0
if
(
i
,
j
)
not
∈
Ω
where W represents a relationship between a pair of words, Ω represents indices of non-zero entries of sparse W, and ij are matrix iterations.
9. The computer of claim 8 , comprising means for enforcing an l 1 regularization with mini-batch shrinking for every predetermined iterations in a stochastic gradient descent.