Boosting scores for ranking items matching a search query
An online system receives a search query from a client device associated with a user and queries a database including item data for a set of items matching the query, in which the set of items is at a retailer location associated with a retailer type and each item is associated with an item category. For each item of the set, a machine learning model is applied to predict a probability of conversion for the user and item and a score is computed based on an expected value, in which the expected value is based on a value associated with the item and the probability. The score for each item is boosted based on the item category, retailer type, or a user segment that is based on the user's historical order data. The items are ranked based on the boosted scores and the ranking is sent to the client device.
1 . A method comprising:
at a computer system comprising a processor and a computer-readable medium:
receiving a search query from a client device associated with a user of an online system;
querying a database comprising item data for a set of items that match at least a portion of the search query, wherein each item of the set of items is associated with an item category;
for each item of the set of items:
computing an initial score for a corresponding item by applying a first machine learning model to a set of attributes of the user and an additional set of attributes of a corresponding item to predict a probability of the user interacting with the corresponding item through a user interface of the client device, wherein the first machine model comprises a neural network;
training a second machine learning model for the item of the corresponding items, wherein the second machine learning model comprises a regression model, wherein the second machine learning model is trained to predict the probability of the user interacting with the corresponding item through a user interface of the client device based on the item category associated with the corresponding item and a user segment associated with the user, wherein the user segment associated with the user is based on historical order data describing the set of orders received from the user;
computing a boosting hyperparameter for the second machine learning model, wherein the boosting hyperparameter is a hyperparameter for the second machine learning model for boosting the initial score for the corresponding item, wherein computing the boosting hyperparameter comprises:
applying the second machine learning model to order data associated with the user using a first candidate boosting hyperparameter of a plurality of candidate boosting hyperparameters;
measuring a performance of the second machine learning model using the first candidate boosting hyperparameter; and
adjusting the first candidate boosting hyperparameter to another boosting hyperparameter based on the measured performance of the second machine learning model; and
boosting the initial score for the corresponding item to generate a boosted score based on the boosting hyperparameter, the item category associated with the corresponding item, and the user segment associated with the user, wherein the user segment associated with the user is based at least in part on the historical order data describing the set of orders received from the user;
ranking the set of items based at least in part on the boosted score for each item;
sending the ranking to the client device associated with the user which causes the client device to generate a user interface in a display area of the client device by:
generating a plurality of user interface elements that each correspond to one item from the set of items, each user interface element for accessing computer functionality of the client device responsive to a selection of the user interface element by the user;
arranging the plurality of user interface elements according to the ranking sent to the client device that is based at least in part on the boosted score for each item from the set of items; and
displaying the plurality of user interface elements that are arranged according to the ranking in the user interface; and
receiving, from the client device, a selection of one of the plurality of user interface elements that is displayed in the user interface,
wherein the computer functionality of the client device that corresponds to the one of the plurality of user interface elements is accessed responsive to receiving the selection.
2 . The method of claim 1 , wherein
the boosting hyperparameter for boosting the initial score for the corresponding item is further determined based on the historical order data describing the set of orders received from the user.
3 . The method of claim 1 , wherein the boosting hyperparameter changes an expected value associated with the corresponding item and the boosting hyperparameter is selected from a group consisting of: a coefficient and an exponent.
4 . The method of claim 3 , wherein the expected value, associated with the corresponding item is based at least in part on the boosting hyperparameter a measure of relevance of the corresponding item to the search query, and a price associated with the corresponding item.
5 . The method of claim 1 , wherein the item category associated with the corresponding item is selected from a group consisting of: groceries, alcohol, tobacco, pharmaceuticals, gifts, clothing, accessories, personal care, books, electronics, furniture, houseware, toys, sporting goods, pet supplies, baby supplies, arts and crafts supplies, office supplies, party supplies, cleaning supplies, home improvement, and appliances.
6 . The method of claim 1 , wherein the user segment associated with the user is selected from a group consisting of: a value-oriented user segment and a premium user segment.
7 . The method of claim 6 , wherein the user segment associated with the user is determined based at least in part on one or more features associated with the user selected from a group consisting of: an average total amount spent on each order of the set of orders received from the user, an average percentage of discounted items included in each order of the set of orders received from the user, an average amount spent on each item included in each order of the set of orders received from the user, an average number of items associated with at least a threshold price included in each order of the set of orders received from the user, an average number of items associated with less than a threshold price included in each order of the set of orders received from the user, an average number of organic items included in each order of the set of orders received from the user, a frequency with which the user placed one or more orders included among the set of orders, and one or more retailer types associated with the set of orders received from the user.
8 . The method of claim 7 , further comprising:
generating a plurality of data points associated with a plurality of users of the online system, wherein a data point is generated based at least in part on the one or more features associated with a corresponding user of the online system;
classifying each of the plurality of data points into a user segment of a plurality of user segments based at least in part on a clustering algorithm; and
determining the user segment associated with the user based at least in part on a classification of a data point associated with the user.
9 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:
receive a search query from a client device associated with a user of an online system;
query a database comprising item data for a set of items that match at least a portion of the search query, wherein the set of items is included among an inventory of a retailer location, the retailer location is associated with a retailer type, and each item of the set of items is associated with an item category;
for each item of the set of items:
computing an initial score for a corresponding item by applying a first machine learning model to a set of attributes of the user and an additional set of attributes of a corresponding item to predict a probability of the user interacting with the corresponding item through a user interface of the client device, wherein the first machine model comprises a neural network;
train a second machine learning model for the item of the corresponding items, wherein the second machine learning model comprises a regression model, wherein the second machine learning model is trained to predict the probability of the user interacting with the corresponding item through a user interface of the client device based on the item category associated with the corresponding item and a user segment associated with the user, wherein the user segment associated with the user is based on historical order data describing the set of orders received from the user;
compute a boosting hyperparameter for the second machine learning model, wherein the boosting hyperparameter is a hyperparameter for the second machine learning model for boosting the initial score for the corresponding item, wherein computing the boosting hyperparameter comprises;
applying the second machine learning model to order data associated with the user using a first candidate boosting hyperparameter of a plurality of candidate boosting hyperparameters;
measuring a performance of the second machine learning model using the first candidate boosting hyperparameter; and
adjusting the first candidate boosting hyperparameter to another boosting hyperparameter based on the measured performance of the second machine learning model; and
boost the initial score for the corresponding item to generate a boosted score based on the boosting hyperparameter the item category associated with the corresponding item, and the user segment associated with the user, wherein the user segment associated with the user is based at least in part on the historical order data describing the set of orders received from the user;
rank the set of items based at least in part on the boosted score for each item;
send the ranking to the client device associated with the user which causes the client device to generate a user interface in a display area of the client device by:
generating a plurality of user interface elements that each correspond to one item from the set of items, each user interface element for accessing computer functionality of the client device responsive to a selection of the user interface element by the user;
arranging the plurality of user interface elements according to the ranking sent to the client device that is based at least in part on the boosted score for each item from the set of items; and
displaying the plurality of user interface elements that are arranged according to the ranking in the user interface; and
receive, from the client device, a selection of one of the plurality of user interface elements that is displayed in the user interface,
wherein the computer functionality of the client device that corresponds to the one of the plurality of user interface elements is accessed responsive to receiving the selection.
10 . The computer program product of claim 9 , wherein
the boosting hyperparameter for boosting the initial score for the corresponding item is further determined based on the historical order data describing the set of orders received from the user.
11 . The computer program product of claim 9 , wherein the boosting hyperparameter changes an expected value associated with the corresponding item and the boosting hyperparameter is selected from a group consisting of: a coefficient and an exponent.
12 . The computer program product of claim 11 , wherein the expected value associated with the corresponding item is based at least in part on the boosting hyperparameter, a measure of relevance of the corresponding item to the search query, and a price associated with the corresponding item.
13 . The computer program product of claim 9 , wherein the item category associated with the corresponding item is selected from a group consisting of: groceries, alcohol, tobacco, pharmaceuticals, gifts, clothing, accessories, personal care, books, electronics, furniture, houseware, toys, sporting goods, pet supplies, baby supplies, arts and crafts supplies, office supplies, party supplies, cleaning supplies, home improvement, and appliances.
14 . The computer program product of claim 9 , wherein the retailer type associated with the retailer location is selected from a group consisting of: traditional grocery store, organic grocery store, convenience store, warehouse club store, specialty store, department store, discount store, premium store, and outlet store.
15 . The computer program product of claim 9 , wherein the user segment associated with the user is selected from a group consisting of: a value-oriented user segment and a premium user segment.
16 . The computer program product of claim 15 , wherein the user segment associated with the user is determined based at least in part on one or more features associated with the user selected from a group consisting of: an average total amount spent on each order of the set of orders received from the user, an average percentage of discounted items included in each order of the set of orders received from the user, an average amount spent on each item included in each order of the set of orders received from the user, an average number of items associated with at least a threshold price included in each order of the set of orders received from the user, an average number of items associated with less than a threshold price included in each order of the set of orders received from the user, an average number of organic items included in each order of the set of orders received from the user, a frequency with which the user placed one or more orders included among the set of orders, and one or more retailer types associated with the set of orders received from the user.
17 . A computer system comprising:
a processor;
and a non-transitory computer readable storage medium storing instructions that, when executed by the processor, perform actions comprising:
receiving a search query from a client device associated with a user of an online system;
querying a database comprising item data for a set of items that match at least a portion of the search query, wherein each item of the set of items is associated with an item category;
for each item of the set of items:
computing an initial score for a corresponding item by applying a first machine learning model to a set of attributes of the user and an additional set of attributes of a corresponding item to predict a probability of the user interacting with the corresponding item through a user interface of the client device, wherein the first machine model comprises a neural network;
training a second machine learning model for the item of the corresponding items, wherein the second machine learning model comprises a regression model, wherein the second machine learning model is trained to predict the probability of the user interacting with the corresponding item through a user interface of the client device based on the item category associated with the corresponding item and a user segment associated with the user, wherein the user segment associated with the user is based on historical order data describing the set of orders received from the user;
computing a boosting hyperparameter for the second machine learning model, wherein the boosting hyperparameter is a hyperparameter for the second machine learning model for boosting the initial score for the corresponding item, wherein computing the boosting hyperparameter comprises:
applying the second machine learning model to order data associated with the user using a first candidate boosting hyperparameter of a plurality of candidate boosting hyperparameters;
measuring a performance of the second machine learning model using the first candidate boosting hyperparameter; and
adjusting the first candidate boosting hyperparameter to another boosting hyperparameter based on the measured performance of the second machine learning model; and
boosting the initial score for the corresponding item to generate a boosted score based on the boosting hyperparameter, the item category associated with the corresponding item, and the user segment associated with the user, wherein the user segment associated with the user is based at least in part on the historical order data describing the set of orders received from the user;
ranking the set of items based at least in part on the boosted score for each item;
sending the ranking to the client device associated with the user which causes the client device to generate a user interface in a display area of the client device by:
generating a plurality of user interface elements that each correspond to one item from the set of items, each user interface element for accessing computer functionality of the client device responsive to a selection of the user interface element by the user;
arranging the plurality of user interface elements according to the ranking sent to the client device that is based at least in part on the boosted score for each item from the set of items; and
displaying the plurality of user interface elements that are arranged according to the ranking in the user interface; and
receiving, from the client device, a selection of one of the plurality of user interface elements that is displayed in the user interface,
wherein the computer functionality of the client device that corresponds to the one of the plurality of user interface elements is accessed.