Query dependent threshold generation for search result filtering
Query dependent threshold generation for search result filtering is described. In one or more implementations, a user query entered via a search platform is received, and in response, items are retrieved from a storage device based on the user query. Using a first machine learning model, relevance scores are generated for the items, and the relevance scores represent degrees of relevance of respective items with respect to the user query. Using a second machine learning model, a relevance threshold is generated for the user query based on one or more features of the user query. The items are filtered based on the relevance scores and the relevance threshold, and the filtered items are communicated over a network for display in a user interface of the search platform.
1 . A method implemented by at least one computing device, the method comprising:
receiving a user query entered via a search platform;
retrieving, from a storage device, items based on the user query;
generating, using a first machine learning model, a relevance threshold for the user query based on one or more features of the user query which capture a degree of specificity of the user query;
distributing the items across a plurality of database shards hosted by different servers;
processing, by the different servers, the items in parallel across the plurality of database shards by:
generating, using a second machine learning model, relevance scores for the items representing degrees of relevance of respective items with respect to the user query; and
filtering the items based on the relevance scores and the relevance threshold; and
communicating, over a network, the filtered items for display in a user interface of the search platform.
2 . The method of claim 1 , wherein generating the relevance threshold includes:
generating, using a first function of the first machine learning model, an intermediate relevance threshold based on the one or more features; and
generating, using a second function of the first machine learning model, the relevance threshold by reducing the intermediate relevance threshold by a relaxation factor.
3 . The method of claim 2 , further comprising:
receiving training data including a plurality of training samples, each training sample including a training query, and training items having been engaged with by users of the search platform responsive to the training query; and
training the first function and the second function of the first machine learning model using the training data.
4 . The method of claim 3 , wherein training the first function includes:
generating, using the second machine learning model, training relevance scores for the training items of a training sample of the plurality of training samples;
generating, using the first function, a predicted intermediate relevance threshold based on the one or more features of the training query of the training sample; and
training the first function based on a comparison of a target value of the training relevance scores and the predicted intermediate relevance threshold.
5 . The method of claim 4 , wherein training the second function includes:
repeating the generating the training relevance scores and the generating the predicted intermediate relevance threshold for each of the plurality of training samples;
computing, for each of the plurality of training samples, a difference between the predicted intermediate relevance threshold and the target value; and
determining, as the relaxation factor, a function of a standard deviation of the differences.
6 . The method of claim 4 , wherein the target value is a minimum value of the training relevance scores.
7 . The method of claim 4 , further comprising ranking the training items for display in the user interface in a display order, wherein the target value is a weighted sum of the training relevance scores having weights determined based on the display order.
8 . The method of claim 3 , further comprising tuning the relaxation factor using the training data.
9 . The method of claim 1 , wherein the one or more features include a number of the items retrieved based on the user query, a number of categories to which the items belong, a number of tokens in the user query, constraints associated with the user query, and one or more categories into which the user query is classified.
10 . The method of claim 1 , wherein filtering the items includes:
identifying relevant items having one or more of the relevance scores that exceed the relevance threshold;
ranking the items in a display order for display in the user interface, the display order including the relevant items ranked above irrelevant items having the relevance scores that are below the relevance threshold; and
communicating, over the network, the items for display in the user interface in the display order.
11 . The method of claim 10 , wherein ranking the items includes:
determining a first ranking of the items, resulting in one or more top-ranked subsets of the items;
determining a second ranking of the relevant items in the one or more top-ranked subsets; and
determining the display order based on the first ranking and the second ranking.
12 . The method of claim 1 , wherein filtering the items includes discarding irrelevant items having the relevance scores that are below the relevance threshold.
13 . A non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including:
receiving a user query entered via a search platform;
retrieving, from a storage device, items based on the user query;
generating, using a first machine learning model, a relevance threshold for the user query based on one or more features of the user query which capture a degree of specificity of the user query;
distributing the items across a plurality of database shards hosted by different servers;
processing, by the different servers, the items in parallel across the plurality of database shards by:
generating, using a second machine learning model, relevance scores for the items representing degrees of relevance of respective items with respect to the user query; and
filtering the items based on the relevance scores and the relevance threshold; and
communicating, over a network, the filtered items for display in a user interface of the search platform.
14 . The non-transitory computer-readable media of claim 13 , the operations further including:
receiving a training sample including a training query, and training items having been engaged with by users of the search platform responsive to the training query;
generating, using the second machine learning model, training relevance scores for the training items;
generating, using the first machine learning model, a predicted relevance threshold based on the one or more features of the training query; and
training the first machine learning model based on a minimum value of the training relevance scores and the predicted relevance threshold.
15 . The non-transitory computer-readable media of claim 14 , wherein training the first machine learning model includes:
determining a delta value based on the training relevance scores;
determining a target relevance threshold by reducing the minimum value by the delta value; and
training the first machine learning model based on a comparison of the predicted relevance threshold and the target relevance threshold.
16 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes computing, as the delta value, a function of a standard deviation of the training relevance scores.
17 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes:
determining differences between the minimum value and the training relevance scores; and
computing, as the delta value, a function of a standard deviation of the differences.
18 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes:
determining differences between the minimum value and the training relevance scores; and
computing, as the delta value, a function of a lower bound of a confidence interval of the differences.
19 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes computing, as the delta value, a function of a fraction distance between a highest value of the training relevance scores and a lowest value of the training relevance scores.
20 . A system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
receive a user query entered via a search platform;
retrieve, from a storage device, items based on the user query;
generate, using a first machine learning model, a relevance threshold for the user query based on one or more features of the user query which capture a degree of specificity of the user query;
distribute the items across a plurality of database shards hosted by different servers;
process, by the different servers, the items in parallel across the plurality of database shards by:
generating, using a second machine learning model, relevance scores for the items representing degrees of relevance of respective items with respect to the user query; and
filtering the items based on the relevance scores and the relevance threshold; and
communicate, over a network, the filtered items for display in a user interface of the search platform.