Ranking documents based on large data sets
View Patent ↗A system ranks documents based, at least in part, on a ranking model. The ranking model may be generated to predict the likelihood that a document will be selected. The system may receive a search query and identify documents relating to the search query. The system may then rank the documents based, at least in part, on the ranking model and form search results for the search query from the ranked documents.
1. A computer-implemented method of ranking search results based on a likelihood of user selection, the method comprising:
receiving a search query from a user;
obtaining a plurality of search results that satisfy the search query, wherein each search result identifies a respective document of a plurality of documents;
identifying a respective condition for each document of the plurality of documents, wherein each condition comprises one or more features of the user, the search query, and the document;
obtaining a ranking model that produces a score for a particular document given a particular condition for the particular document, the score representing a likelihood that the user will select the particular document when identified by a search result provided in response to the search query, the ranking model being trained on training instances that each identify a first document selected by a particular user when the first document was identified in search results provided to the particular user in response to a particular search query;
using the ranking model to compute a respective score for each document of the plurality of documents; and
ranking the plurality of search results according to the respective computed score for each document of the plurality of documents.
2. The method of claim 1 , wherein each training instance identifies a condition comprising one or more features of the particular user and one or more features of the particular search query.
3. The method of claim 1 , wherein each training instance identifies one or more second documents that the particular user did not select when the one or more second documents were identified by the search results provided to the particular user in response to the particular search query.
4. The method of claim 1 , wherein each training instance includes data representing a position of the selected first document in an order of the search results provided to the particular user in response to the particular query.
5. The method of claim 1 , wherein each training instance includes data representing a previously computed score for the selected first document.
6. The method of claim 1 , wherein each training instance comprises data representing a number of documents ranked above the selected first document in the search results provided to the particular user in response to the particular search query.
7. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving a search query from a user;
obtaining a plurality of search results that satisfy the search query, wherein each search result identifies respective document of a plurality of documents;
identifying a respective condition for each document of the plurality of documents, wherein each condition comprises one or more features of the user, the search query, and the document;
obtaining a ranking model that produces a score for a particular document given a particular condition for the particular document, the score representing a likelihood that the user will select the particular document when identified by a search result provided in response to the search query, the ranking model being trained on training instances that each identify a first document selected by a particular user when the first document was identified in search results provided to the particular user in response to a particular search query;
using the ranking model to compute a respective score for each document of the plurality of documents; and
ranking the plurality of search results according to the respective computed score for each document of the plurality of documents.
8. The computer program product of claim 7 , wherein each training instance identifies a condition comprising one or more features of the particular user and one or more features of the particular search query.
9. The computer program product of claim 7 , wherein each training instance identifies one or more second documents that the particular user did not select when the one or more second documents were identified by the search results provided to the particular user in response to the particular search query.
10. The computer program product of claim 7 , wherein each training instance includes data representing a position of the selected first document in an order of the search results provided to the particular user in response to the particular query.
11. The computer program product claim 7 , wherein each training instance includes data representing a previously computed score for the selected first document.
12. The computer program product of claim 7 , wherein each training instance comprises data representing a number of documents ranked above the selected first document in the search results provided to the particular user in response to the particular search query.
13. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a search query from a user;
obtaining a plurality of search results that satisfy the search query, wherein each search result identifies a respective document of a plurality of documents;
identifying a respective condition for each document of the plurality of documents, wherein each condition comprises one or more features of the user, the search query, and the document;
obtaining a ranking model that produces a score for a particular document given a particular condition for the particular document, the score representing a likelihood that the user will select the particular document when identified by a search result provided in response to the search query, the ranking model being trained on training instances that each identify a first document selected by a particular user when the first document was identified in search results provided to the particular user in response to a particular search query;
using the ranking model to compute a respective score for each document of the plurality of documents; and
ranking the plurality of search results according to the respective computed score for each document of the plurality of documents.
14. The system of claim 13 , wherein each training instance identifies a condition comprising one or more features of the particular user and one or more features of the particular search query.
15. The system of claim 13 , wherein each training instance identifies one or more second documents that the particular user did not select when the one or more second documents were identified by the search results provided to the particular user in response to the particular search query.
16. The system of claim 13 , wherein each training instance includes data representing a position of the selected first document in an order of the search results provided to the particular user in response to the particular query.
17. The system claim 13 , wherein each training instance includes data representing a previously computed score for the selected first document.
18. The system of claim 13 , wherein each training instance comprises data representing a number of documents ranked above the selected first document in the search results provided to the particular user in response to the particular search query.