Method and system for comparing documents based on different document-similarity calculation methods using adaptive weighting
One embodiment provides a system for comparing documents based on different document-similarity calculation methods using adaptive weighting. During operation, the system receives at least two document-similarity values associated with two documents, wherein the document-similarity values are calculated by different document-similarity calculation methods. The system then determines the weight of a respective document-similarity calculation method for each of the two documents, as well as a weight-combination function for calculating a combined weight of the respective document-similarity calculation method associated with the two documents. Next, the system generates a combined similarity value based on the document-similarity values and the weight-combination function.
1. A computer-executable method for comparing documents, the method comprising:
receiving, by a computer, a number of document-similarity values s i (a,b) associated with two documents, the document-similarity values being calculated by different document-similarity calculation methods, which are indexed by i;
determining weights (α i , β i ) of a respective document-similarity calculation method i for each of the two documents a and b respectively, wherein α i corresponds to document a and β i corresponds to document b, wherein determining the weights of a document-similarity calculation method involves:
initializing the weights; and
updating the weights based on user feedback, wherein updating the weights of the document-similarity calculation method for each document involves applying a learning algorithm to the weights based on user feedback;
determining a weight-combination function ƒ(α i ,β i ) for calculating a combined weight of the respective document-similarity calculation method i associated with the two documents, wherein the weight-combination function accounts for the direction of comparison between the documents a and b; and
generating, by the computer, a combined similarity value S(a,b) based on the document-similarity values calculated by k different document-similarity calculation methods and the weight-combination function, given by:
S
(
a
,
b
)
=
∑
i
=
1
k
f
(
α
i
,
β
i
)
·
s
i
(
a
,
b
)
.
2. The method of claim 1 , wherein the document-similarity calculation methods include one or more of: text-based, visual-based, usage-based, and social network-based document-similarity calculation methods.
3. The method of claim 1 , wherein the weight of the document-similarity calculation method for each document is initialized based at least on one of: document type, document location, document structure, document usage, and weights of related documents.
4. The method of claim 1 , wherein the weight-combination function is a real-valued function that calculates an average, a minimum, or a maximum of the weights.
5. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for comparing documents, the method comprising:
receiving a number of document-similarity values s i (a,b) associated with two documents, the document-similarity values being calculated by different document-similarity calculation methods, which are indexed by i;
determining weights (α i ,β i ) of a respective document-similarity calculation method i for each of the two documents a and b respectively, wherein α i corresponds to document a and β i corresponds to document b, wherein determining the weights of a document-similarity calculation method involves:
initializing the weights; and
updating the weights based on user feedback, wherein updating the weights of the document-similarity calculation method for each document involves applying a learning algorithm to the weights based on user feedback;
determining a weight-combination function ƒ(α i ,β i ) for calculating a combined weight of the respective document-similarity calculation method i associated with the two documents, wherein the weight-combination function accounts for the direction of comparison between the documents a and b; and
generating a combined similarity value S(a,b) based on the document-similarity values calculated by k different document-similarity calculation methods and the weight-combination function, given by:
S
(
a
,
b
)
=
∑
i
=
1
k
f
(
α
i
,
β
i
)
·
s
i
(
a
,
b
)
.
6. The computer-readable storage medium of claim 5 , wherein the document-similarity calculation methods include one or more of: text-based, visual-based, usage-based, and social network-based document-similarity calculation methods.
7. The computer-readable storage medium of claim 5 , wherein the weight of the document-similarity calculation method for each document is initialized based at least on one of: document type, document location, document structure, document usage, and weights of related documents.
8. The computer-readable storage medium of claim 5 , wherein the weight-combination function is a real-valued function that calculates an average, a minimum, or a maximum of the weights.
9. A system for estimating a similarity level between documents, comprising:
at least one hardware processor couple to at least one memory for estimating the similarity level between documents;
a receiving mechanism configured to receive a number of document-similarity values s i (a,b) associated with two documents, the document-similarity values being calculated by different document-similarity calculation methods, which are indexed by i;
a determination mechanism configured to determine:
weights (α i ,β i ) of a respective document-similarity calculation method i for each of the two documents a and b respectively, wherein α i corresponds to document a and β i corresponds to document b, wherein determining the weights of a document-similarity calculation method involves:
initializing the weights; and
updating the weights based on user feedback, wherein updating the weights of the document-similarity calculation method for each document involves applying a learning algorithm to the weights based on user feedback; and
a weight-combination function ƒ(α i ,β i ) for calculating a combined weight of the respective document-similarity calculation method i associated with the two documents, wherein the weight-combination function accounts for the direction of comparison between the documents a and b; and
generating a combined similarity value S(a,b) based on the document-similarity values calculated by k different document-similarity calculation methods and the weight-combination function, given by:
S
(
a
,
b
)
=
∑
i
=
1
k
f
(
α
i
,
β
i
)
·
s
i
(
a
,
b
)
.
10. The system of claim 9 , wherein the document-similarity calculation methods include one or more of: text-based, visual-based, usage-based, and social network-based document-similarity calculation methods.
11. The system of claim 9 , wherein the weight of the document-similarity calculation method for each document is initialized based at least on one of: document type, document location, document structure, document usage, and weights of related documents.
12. The system of claim 9 , wherein the weight-combination function is a real-valued function that calculates an average, a minimum, or a maximum of the weights.