Preliminary ranker for scoring matching documents
The technology described herein provides for preliminary ranking of matching documents for a search query. A preliminary ranker uses score tables for scoring each matching document based on its relevant to a search query. The score table for a document stores pre-computed data used to derive a frequency of terms and other information in the document. The preliminary ranker uses the score table for each matching document and the terms form the search query to determine a score for each matching document. The lowest scoring documents are removed from further consideration by a final ranker.
1. A computer-implemented method for scoring a plurality of documents based on relevancy to a search query, the method comprising:
accessing a table that is associated with a document found to be relevant to at least a portion of the search query, wherein the table stores pre-computed data used to derive a frequency of each term of a subset of terms in the document, and wherein each term of the subset of terms occurs in the document more than once;
algorithmically determining the frequency of at least one term that corresponds to the search query; and
based at least on the frequency and other data associated with the document and the search query, computing a score of the document in relation to the at least one term that corresponds to the search query.
2. The method of claim 1 , wherein the score is further based on one or more real-time components that are computed in real-time, the one or more real-time components corresponding to the document and the at least one term.
3. The method of claim 2 , wherein the one or more real-time components comprise a location of the at least one term in the search query and a position of each of the at least one term in relation to one another in the search query.
4. The method of claim 1 , further comprising receiving identifications of a plurality of documents found to be potentially relevant to the search query based on a keyword match.
5. The method of claim 1 , wherein less than half of all terms found in the document is included in the subset of terms.
6. The method of claim 1 , wherein the computing of the score is further based on an infrequency of the at least one term in a corpus of documents.
7. The method of claim 1 , wherein the computing of the score is further based on one or more portions of the document in which the at least one term is located.
8. The method of claim 1 , wherein the computing of the score is further based on one or more of a comparison of a language of the document to the language of the search query or a comparison of a geographical location associated with the document to the geographical location associated with the search query.
9. The method of claim 1 , wherein the computing of the score further comprises:
computing scores for each of one or more pre-computed components, wherein the frequency of the at least one term is one of the one or more pre-computed components; and
computing scores for each of one or more real-time components that are computed after the search query is received.
10. The method of claim 1 , wherein the frequency is computed using data stored in the table, the data stored in the table comprising a pre-computed inverse document frequency of the at least one term.
11. A computer-implemented method comprising:
accessing a table having a plurality of slots that store data associated with a document;
for a first slot of the table, comparing a portion of a first hash key associated with a first term whose corresponding data is stored in the first slot with a portion of a second hash key associated with a second term whose data is being considered to be added to the first slot;
if the portion of the first hash key matches the portion of the second hash key,
(1) determining, from the data in the table, that a frequency of the second term in the document is greater than the frequency of the first term in the document, and
(2) storing data associated with the frequency of the second term in the first slot of the table; and
if the portion of the first hash key does not match the portion of the second hash key, not storing data corresponding to the second term in the first slot of the table.
12. The method of claim 11 , wherein if the portion of the first hash key matches the portion of the second hash key, further comprising removing data associated with the frequency of the first term from the first slot of the table.
13. The method of claim 12 , further comprising if the portion of the first hash key does not match the portion of the second hash key, comparing the portion of the second hash key with a portion of a third hash key that is associated with a third term whose corresponding data is stored in a second slot of the table.
14. The method of claim 13 , further comprising if the portion of the second hash key matches the portion of the third hash key, determining, from the data in the table, whether a frequency of the second term in the document is greater than the frequency of the third term in the document.
15. The method of claim 14 , wherein if the frequency of the second term in the document is greater than the frequency of the third term in the document, further comprising storing data associated with the frequency of the second term in the second slot of the table.
16. The method of claim 13 , further comprising if the portion of the second hash key does not match the portion of the third hash key, not storing data corresponding to the second term in the second slot of the table.
17. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform a method for scoring a plurality of documents based on relevancy to a search query, the method comprising:
accessing a table that stores data corresponding to a document, the data used to derive one or more pre-computed components that contribute to a score of the document in relation to the search query, the one or more pre-computed components comprising: (1) a frequency of one or more terms in the document, wherein the one or more terms are found in the document at least two times, and (2) at least one portion of the document in which the one or more terms are located;
computing scores for each of the one or more pre-computed components including the frequency of the one or more terms in the document and the at least one portion of the document in which the one or more terms are located;
computing scores for each of one or more real-time components that are computed after the search query is entered; and
computing a final score for the document in relation to the search query based on the scores for the one or more pre-computed components and the one or more real-time components.
18. The method of claim 17 , wherein the least one portion of the document is one or more of an anchor stream, a title stream, a body stream, or a URL stream.
19. The method of claim 17 , wherein the frequency of the one or more terms is stored in the table as a pre-computed inverse document frequency.
20. The method of claim 17 , further comprising accessing a click table that stores click data representing how often the document is selected by other users for the one or more terms.