User click modelling in search queries
A method for ranking documents in search results includes defining a first training data set, the first training data set including, for each of a plurality of user queries, information respective of a document selected by a user from results responsive to the query and information respective of one or more documents within an observation window after the selected document in the results, and defining a second training data set, the second training data set including, for each of the plurality of user queries, information respective of the selected document. The method further includes training a first machine learning model with the first training data set, training a second machine learning model with the second training data set, and ranking documents of a further search result set according to the output of the first machine learning model and the output of the second machine learning model.
1. A method for ranking documents in search results, the method comprising:
retrieving a plurality of search result sets, each search result set being associated with a user query and comprising an ordered plurality of documents;
defining a training data set based on the plurality of search result sets by, for each search result set:
determining an observation window including a pre-defined number of documents ordered after a responsive document from the search result set, the responsive document representative of a document selected by a user from the search result set; and
discarding documents from the search result set that are ordered below the pre-defined number of documents after the responsive document;
training a machine learning model via the training data set;
receiving a further user query;
presenting a list of responsive documents ranked by the trained machine learning model;
receiving indication of a responsive document from the presented list of responsive documents;
processing the list of responsive documents to discard documents from the list of responsive documents that are outside of the observation window from the indicated responsive document; and
adding the processed list of responsive documents to the training data set,
wherein, by not including the discarded documents, the training data set reduces a bias representative of a user selection of the responsive document over the discarded documents at the machine learning model.
2. The method of claim 1 , wherein training the machine learning model comprises:
conducting reinforcement learning on the machine learning model to maximize a prediction accuracy of the machine learning model.
3. The method of claim 1 , wherein the observation window includes between one and three documents after the responsive document.
4. The method of claim 1 , wherein:
the plurality of search result sets comprises:
a plurality of previous search result sets; and
one or more current search result sets; and
training the machine learning model with the defined training data set comprises:
batch training the machine learning model according to the previous search result sets; and
conducting reinforcement learning on the machine learning model according to the one or more current search result sets.
5. The method of claim 4 , wherein conducting reinforcement learning comprises maximizing a prediction accuracy of the machine learning model.
6. The method of claim 1 , wherein presenting the list of responsive documents comprises displaying the ranked further search result set on a user device.
7. A system comprising:
a non-transitory, computer-readable medium storing instructions; and
a processor configured to execute the instructions to:
retrieve a plurality of search result sets, each search result set being associated with a user query and comprising an ordered plurality of documents;
define a training data set based on the plurality of search result sets by, for each search result set:
determine an observation window including a pre-defined number of documents ordered after a responsive document from the search result set, the responsive document representative of a document selected by a user from the search result set; and
discard documents from the search result set that are ordered below the pre-defined number of documents after the responsive document;
train a machine learning model via the training data set;
receive a further user query;
present a list of responsive documents ranked by the trained machine learning model;
receiving indication of a responsive document from the presented list of responsive documents;
processing the list of responsive documents to discard documents from the list of responsive documents that are outside of the observation window from the indicated responsive document; and
adding the processed list of responsive documents to the training data set,
wherein, by not including the discarded documents, the training data set reduces a bias representative of a user selection of the responsive document over the discarded documents at the machine learning model.
8. The system of claim 7 , wherein training the machine learning model comprises:
conducting reinforcement learning on the machine learning model to maximize a prediction accuracy of the machine learning model.
9. The system of claim 7 , wherein the observation window includes between one and three documents after the responsive document.
10. The system of claim 7 , wherein:
the plurality of search result sets comprises:
a plurality of previous search result sets; and
one or more current search result sets; and
training the machine learning model with the defined training data set comprises:
batch training the machine learning model according to the previous search result sets; and
conducting reinforcement learning on the machine learning model according to the one or more current search result sets.
11. The system of claim 10 , wherein conducting reinforcement learning comprises maximizing a prediction accuracy of the machine learning model.
12. The system of claim 7 , wherein presenting the list of responsive documents comprises displaying the ranked further search result set on a user device.
13. A method for presenting search results, the method comprising:
retrieving a plurality of search result sets, each search result set being associated with a user query and comprising an ordered plurality of documents;
defining a first training data set based on the plurality of search result sets by, for each search result set:
determining an observation window including a pre-defined number of documents ordered after a responsive document from the search result set, the responsive document representative of a document selected by a user from the search result set; and
discarding documents from the search result set that are ordered below the pre-defined number of documents after the responsive document;
training a first machine learning model via the first training data set;
defining a second training data set based on the responsive document from each of the plurality of search result sets;
training a second machine learning model via the second training data set;
receiving a further user query;
presenting a list of responsive documents ranked according to the first and second trained machine learning models;
receiving indication of a responsive document from the presented list of responsive documents;
processing the list of responsive documents to discard documents from the list of responsive documents that are outside of the observation window from the indicated responsive document; and
adding the processed list of responsive documents to the training data set,
wherein, by not including the discarded documents, the first training data set reduces a bias representative of a user selection of the responsive document over the discarded documents at the first machine learning model.
14. The method of claim 13 , wherein training the first machine learning model and training the second machine learning model comprises:
conducting reinforcement learning on the first machine learning model and the second machine learning model to maximize a reward, the reward comprising a combination of a prediction accuracy of the first machine learning model and a prediction accuracy of the second machine learning model.
15. The method of claim 13 , wherein the observation window includes between one and three documents after the responsive document.
16. The method of claim 13 , wherein:
the plurality of search result sets comprises:
a plurality of previous search result sets; and
one or more current search result sets; and
training the first machine learning model with the first training data set comprises:
batch training the machine learning model according to the previous search result sets; and
conducting reinforcement learning on the machine learning model according to the one or more current search result sets.
17. The method of claim 16 , wherein conducting reinforcement learning comprises maximizing a reward, the reward comprising a combination of a prediction accuracy of the first machine learning model and a prediction accuracy of the second machine learning model.