System and method for measuring the quality of document sets
Systems and methods are described that calculate the interestingness of a set of one or more records in a database, either absolutely (i.e., compared to an overall collection of records) or relative to some other set of records. In one embodiment, the measure is a relative entropy value that has been normalized. Various applications of the measure are described in the context of an information retrieval system. These applications include, for example, guiding query interpretation, guiding view selection and summarization, intelligent ranges, event detection, concept triggers and interpreting user actions, hierarchy discovery, and adaptive data mining.
1. A computer implemented method for comparing the distinctiveness of a plurality of sets within a collection of information, the method comprising:
sampling, by a computer system, from the collection of information to generate at least one set;
establishing, automatically, at least one identifying characteristic within the at least one set;
determining a statistical distribution of the at least one identifying characteristic associated with the at least one set; and
generating, by the computer system, a relative measurement of distinctiveness based on the statistical distribution of the at least one identifying characteristic associated with the at least one set and at least one other set, wherein the generating the relative measure of distinctiveness comprises accounting for a set size of a measured set based on a measurement of distinctiveness for a comparison set and a size for the comparison set, and normalizing the relative measurement of distinctiveness based on the set size of the measured set and the size for the comparison set.
2. The method according to claim 1 , wherein the sampling from the collection of information occurs against a result returned from the collection of information.
3. The method according to claim 1 , wherein the at least one set is the comparison set and the at least one other set comprises the measured set, and wherein the measured set comprises a result returned from the collection of information.
4. The method according to claim 1 , wherein at least one of the plurality of sets is a result produced by interaction with the collection of information.
5. The method according to claim 1 , wherein the sampling to generate the at least one set further comprises generating the at least one set having a size based on a size of the at least one other set.
6. The method according to claim 1 , wherein the at least one set comprises at least one document, wherein the at least one document further comprises a unit of storage of digital data.
7. The method according to claim 1 , wherein the at least one identifying characteristic comprises at least one of at least a portion of: textual information within a document; metadata associated with a document; contextual information associated with a document; non-textual information associated with a document; record information with a database; information associated with a composite entity; or information derivable from a document.
8. The method according to claim 1 , further comprising calculating a statistical distribution for each one of at least one of the identifying characteristic.
9. The method according to claim 1 , wherein the statistical distribution is determined against multiple dimensions.
10. The method according to claim 9 , further comprising determining at least one value associated with at least one set member.
11. The method according to claim 1 , wherein the at least one identifying characteristic comprises at least one facet associated with a document.
12. The method according to claim 1 , further comprising generating a representation of the sampled set, wherein the representation of the sampled set is adapted to statistical manipulation.
13. The method according to claim 1 , further comprising assigning a weight value associated with at least one set member.
14. The method according to claim 1 , wherein the measurement of distinctiveness is determined from at least one function of: relative entropy, Kullback-Leibler divergence, Euclidean distance, Manhattan distance, Hellinger distance, diversity difference, cosine difference, Jaccard distance, Jenson-Shannon divergence, or skew divergence.
15. The method according to claim 1 , wherein the generating the measurement of distinctiveness further comprises:
determining a similarity measure; and
inverting the sense of the similarity measure.
16. A non-transitory computer-readable medium having computer-readable instructions stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to compare the distinctiveness of a plurality of sets generated through interaction with a collection of information, the comparing comprising:
sampling from the collection of information to generate at least one set;
establishing, automatically, at least one identifying characteristic within the at least one set;
determining a statistical distribution of the at least one identifying characteristic associated with elements of the at least one set;
generating a relative measurement of distinctiveness based on the statistical distributions of the at least one identifying characteristic associated the at least one set and at least one other set, wherein the generating the relative measure of distinctiveness comprises accounting for a set size of a measured set based on a measurement of distinctiveness for a comparison set and a set size for the comparison set, and normalizing the relative measurement of distinctiveness based on the set size of the measured set and the size for the comparison set.
17. A system for comparing the distinctiveness of a plurality of sets generated through interaction with a collection of information, the system comprising:
at least one processor operatively connected to a memory adapted to execute system components;
a sampling component configured to sample from the collection of information to generate at least one set, wherein the sampling component is further configured to establish, automatically, at least one identifying characteristic within the at least one set;
an analysis component configured to determine a statistical distribution of at least one identifying characteristic associated with the at least one set;
a measurement component configured to determine a relative measurement of distinctiveness based on the statistical distributions of the at least one identifying characteristic associated with the at least one set and at least one other set, wherein the measurement component is further configured to account for a set size of a measured set based on a measurement of distinctiveness for a comparison set and a set size for the comparison set, and normalize the relative measurement of distinctiveness based on the set size of the measured set and the size for the comparison set.
18. The system according to claim 17 , wherein the sampling component samples against a result returned from the collection of information in response to interaction with a collection of information.
19. The system according to claim 17 , wherein the at least one set is the comparison set and the at least one other set is a result returned from the collection of information.
20. The system according to claim 17 , wherein at least one of the plurality of sets is a result produced by interaction with the collection of information.
21. The system according to claim 17 , wherein the at least one set comprises at least one document, wherein the at least one document further comprises a unit of storage of digital data.
22. The system according to claim 17 , wherein the at least one identifying characteristic comprises at least one of at least a portion of: textual information within a document; metadata associated with a document; contextual information associated with a document; non-textual information associated with a document; record information with a database; information associated with a composite entity; or information derivable from a document.
23. The system according to claim 17 , wherein the analysis component is further configured to determine the statistical distribution against a plurality of identifying characteristics.
24. The system according to claim 17 , further comprising a relation component configured to determine at least one value associated with at least one set member.
25. The system according to claim 17 , wherein the at least one identifying characteristic comprises at least one facet associated with a document.
26. The system according to claim 17 , further comprising a representation component configured to generate a representation of the sampled set adapted to statistical manipulation.
27. The system according to claim 17 , further comprising a weighting component configured to assign a weight value to at least one set member.
28. The system according to claim 27 , wherein the weighting component is further configured to maintain the associated weight value.
29. The system according to claim 17 , wherein the measurement component is further configured to determine the measurement of distinctiveness from at least one function of: relative entropy, Kullback-Leibler divergence, Euclidean distance, Manhattan distance, Hellinger distance, diversity difference, cosine difference, Jaccard distance, Jenson-Shannon divergence, or skew divergence.
30. The system according to claim 17 , wherein the measurement component is further configured to determine a similarity measure, and invert the sense of the similarity measure.