Generating an interface displaying items offered by a warehouse that accounts for predicted availabilities of items determined from a trained model
An online concierge system allows users to purchase items from warehouses and delivers the purchased items to the users. When displaying items offered by a warehouse, the online concierge system predicts an availability of the items at the warehouse using a trained model. When displaying items offered by the warehouse to a user, the online concierge system accounts for the predicted availabilities of different items. For example, the online concierge system determines scores for different items at the warehouse based on relevance to the user and adjusts a score for an item by its predicted availability. The online concierge system uses the adjusted scores for items when displaying items, demoting positions in an interface in which items with lower predicted availabilities are displayed. Additionally, the online concierge system may display a visual indication of a predicted availability of certain items, such as items with less than a threshold predicted availability.
1 . A method comprising:
obtaining, at an online system, an item database describing items at one or more locations;
receiving, at the online system, an identification of the location from a user of the online system;
receiving a search query to identify one or more items offered by the location at the online system;
selecting, by the online system, a set of items offered by the location from the item database for the location, each item of the set associated with information that is at least partially matched by the search query;
applying a machine-learned predictive model to data describing the user and each item of the set to output a likelihood of the user interacting with each item of the set, wherein the machine-learned predictive model is trained by:
accessing historical data comprising historical user interactions with items;
generating embeddings for users and items based on the historical data;
associating each user-item pair with a label indicating whether the user interacted with the item;
applying the machine-learned predictive model to each user-item pair to output a predicted probability that the user will interact with the item; and
adjusting parameters of the machine-learned predictive model to reduce an error between the predicted probability and the label;
generating, by the online system, a relevance score for each item of the set based on an amount of information describing the item matched by the search query and the likelihood of the user including the respective item in an order;
ranking, by the online system, the items of the set based on the relevance scores;
determining, by the online system, a predicted availability of each item of the set using a machine learned availability model that is trained to predict a likelihood that an item is available, wherein the availability model is trained by:
receiving a plurality of attributes associated with a plurality of items,
receiving, for each item of the plurality of items, a label indicating an availability of the item,
applying the availability model to each of the plurality of items to determine a difference between the label and the predicted likelihood of the respective item,
updating the label of the respective item based on the determined difference, and
updating the availability model for each of the plurality of items using the updated labels;
determining, by the online system, a confidence score for each item of the set, wherein determining the confidence score comprises computing a weighted sum of at least the relevance score and the predicted availability of the respective item;
modifying, by the online system, the ranking of the items of the set by adjusting a position in the ranking of the item of the set based on the confidence score for the item of the set;
repositioning, by the online system, visual representations of the items within a graphical user interface according to their modified rankings, such that higher-ranked items are displayed at more prominent locations, wherein repositioning includes promoting or demoting, within the graphical user interface, a position of an item relative to other items based on the confidence score of the item relative to the other items; and
transmitting the user interface from the online system to a client device of the user for display.
2 . The method of claim 1 , wherein the confidence score of the item of the set is further calculated using one or more selected from a group consisting of: a degree with which information describing the item of the set, a measure of specificity of the search query, and a probability of the item of the set being replaced by one or more replacement items.
3 . The method of claim 1 , wherein modifying, by the online system, the ranking of the items of the set by adjusting a position in the ranking of the item of the set based on the confidence score for the item of the set comprises:
retrieving ranges of confidence scores maintained by the online system;
determining a range of confidence scores including the confidence score of the item of the set; and
responsive to the determined range of confidence scores indicating a moderate predicted availability of the item of the set, decreasing the position in the ranking of the item of the set.
4 . The method of claim 3 , wherein decreasing the position in the ranking of the item of the set comprises:
decreasing the position in the ranking of the item of the set by a specific number of positions.
5 . The method of claim 3 , wherein decreasing the position in the ranking of the item of the set comprises:
decreasing the position in the ranking of the item of the set to an alternative position that is lower than a specific position in the ranking.
6 . The method of claim 3 , further comprising:
responsive to the determined range of confidence scores indicating a low predicted availability of the item of the set, decreasing the position in the ranking of the item of the set, decreasing a position of the item of the set in the ranking to a position that is below positions in the ranking of each item of the set having a confidence score in the range of confidence scores indicating the moderate predicted availability and storing an indication in association with the item of the set that the item of the set has a low predicted availability.
7 . The method of claim 6 , further comprising:
displaying information describing the item of the set in the position that is below positions in the ranking of each item of the set having the confidence score in the range of confidence scores indicating the moderate predicted availability and displaying the indication the item of the set has the low predicted availability in conjunction with the information describing the item of the set in the position that is below positions in the ranking of each item of the set having the confidence score in the range of confidence scores indicating the moderate predicted availability.
8 . The method of claim 3 , further comprising:
responsive to the determined range of confidence scores indicating a low predicted availability of the item of the set, removing the item of the set from the ranking.
9 . The method of claim 3 , wherein decreasing the position in the ranking of the item of the set comprises:
determining that the position in the ranking of the item of the set equals or exceeds a threshold position; and
in response to determining that the position in the ranking of the item of the set equals or exceeds a threshold position, maintaining the position in the ranking of the item of the set.
10 . A system comprising:
one or more processors; and
a non-transitory computer readable medium having instructions encoded thereon, that when executed by the one or more processors, cause the one or more processors to perform steps including:
obtaining, at an online system, an item database describing items at one or more locations;
receiving, at the online system, an identification of the location from a user of the online system;
receiving a search query to identify one or more items offered by the location at the online system;
selecting, by the online system, a set of items offered by the location from the item database for the location, each item of the set associated with information that is at least partially matched by the search query;
applying a machine-learned predictive model to data describing the user and each item of the set to output a likelihood of the user interacting with each item of the set, wherein the machine-learned predictive model is trained by:
accessing historical data comprising historical user interactions with items;
generating embeddings for users and items based on the historical data;
associating each user-item pair with a label indicating whether the user interacted with the item;
applying the machine-learned predictive model to each user-item pair to output a predicted probability that the user will interact with the item; and
adjusting parameters of the machine-learned predictive model to reduce an error between the predicted probability and the label;
generating, by the online system, a relevance score for each item of the set based on an amount of information describing the item matched by the search query and the likelihood of the user including the respective item in an order;
ranking, by the online concierge-system, the items of the set based on the relevance scores;
determining, by the online system, a predicted availability of each item of the set using a machine learned availability model that is trained to predict a likelihood that an item is available, wherein the availability model is trained by:
receiving a plurality of attributes associated with a plurality of items,
receiving, for each item of the plurality of items, a label indicating an availability of the item,
applying the availability model to each of the plurality of items to determine a difference between the label and the predicted likelihood of the respective item,
updating the label of the respective item based on the determined difference, and
updating the availability model for each of the plurality of items using the updated labels;
determining, by the online system, a confidence score for each item of the set, wherein determining the confidence score comprises computing a weighted sum of at least the relevance score and the predicted availability of the respective item;
modifying, by the online system, the ranking of the items of the set by adjusting a position in the ranking of the item of the set based on the confidence score for the item of the set;
repositioning, by the online system, visual representations of the items within a graphical user interface according to their modified rankings, such that higher- ranked items are displayed at more prominent locations, wherein repositioning includes promoting or demoting, within the graphical user interface, a position of an item relative to other items based on the confidence score of the item relative to the other items; and
transmitting the user interface from the online system to a client device of the user for display.
11 . The system of claim 10 , wherein the confidence score of the item of the set is further calculated using one or more selected from a group consisting of: a degree with which information describing the item of the set, a measure of specificity of the search query, and a probability of the item of the set being replaced by one or more replacement items.
12 . The system of claim 10 , wherein the confidence score for the item of the set comprises a weighted combination of the relevance score for the item of the set and the confidence score for each item of the set.
13 . A computer program product comprising a non-transitory computer readable medium having instructions encoded thereon, that when executed by a processor, cause the processor to:
obtain, at an online system, an item database describing items at one or more locations;
receive, at the online system, an identification of the location from a user of the online system;
receive a search query to identify one or more items offered by the location at the online system;
select, by the online system, a set of items offered by the location from the item database for the location, each item of the set associated with information that is at least partially matched by the search query;
apply a machine-learned predictive model to data describing the user and each item of the set to output a likelihood of the user interacting with each item of the set, wherein the machine-learned predictive model is trained by:
accessing historical data comprising historical user interactions with items;
generating embeddings for users and items based on the historical data;
associating each user-item pair with a label indicating whether the user interacted with the item;
applying the machine-learned predictive model to each user-item pair to output a predicted probability that the user will interact with the item; and
adjusting parameters of the machine-learned predictive model to reduce an error between the predicted probability and the label;
generate, by the online system, a relevance score for each item of the set based on an amount of information describing the item matched by the search query and the likelihood of the user including the respective item in an order;
rank, by the online system, the items of the set based on the relevance scores;
determine, by the online system, a predicted availability of each item of the set using a machine learned availability model that is trained to predict a likelihood that an item is available, wherein the availability model is trained by:
receiving a plurality of attributes associated with a plurality of items,
receiving, for each item of the plurality of items, a label indicating an availability of the item,
applying the availability model to each of the plurality of items to determine a difference between the label and the predicted likelihood of the respective item,
updating the label of the respective item based on the determined difference, and
updating the availability model for each of the plurality of items using the updated labels;
determine, by the online system, a confidence score for each item of the set, wherein determining the confidence score comprises computing a weighted sum of at least the relevance score and the predicted availability of the respective item;
modify, by the online system, the ranking of the items of the set by adjusting a position in the ranking of the item of the set based on the confidence score for the item of the set;
reposition, by the online system, visual representations of the items within a graphical user interface according to their modified rankings, such that higher-ranked items are displayed at more prominent locations, wherein repositioning includes promoting or demoting, within the graphical user interface, a position of an item relative to other items based on the confidence score of the item relative to the other items; and
transmit the user interface from the online system to a client device of the user for display.
14 . The computer program product of claim 13 , wherein the confidence score of the item of the set is further calculated using one or more selected from a group consisting of: a degree with which information describing the item of the set, a measure of specificity of the search query, and a probability of the item of the set being replaced by one or more replacement items.
15 . The computer program product of claim 13 , wherein the instructions to modify, by the online system, the ranking of the items of the set by adjusting a position in the ranking of the item of the set based on the confidence score for the item of the set further cause the processor to:
retrieve ranges of confidence scores maintained by the online system;
determine a range of confidence scores including the confidence score of the item of the set; and
responsive to the determined range of confidence scores indicating a moderate predicted availability of the item of the set, decrease the position in the ranking of the item of the set.
16 . The computer program product of claim 15 , wherein the instructions to decrease the position in the ranking of the item of the set further cause the processor to:
decrease the position in the ranking of the item of the set by a specific number of positions.
17 . The computer program product of claim 15 , wherein the instructions to decrease the position in the ranking of the item of the set further cause the processor to:
decrease the position in the ranking of the item of the set to an alternative position that is lower than a specific position in the ranking.
18 . The computer program product of claim 15 , wherein the instructions, when executed by the processor, further cause the processor to:
responsive to the determined range of confidence scores indicating a low predicted availability of the item of the set, decrease the position in the ranking of the item of the set, decrease a position of the item of the set in the ranking to a position that is below positions in the ranking of each item of the set having a confidence score in the range of confidence scores indicating the moderate predicted availability and storing an indication in association with the item of the set that the item of the set has a low predicted availability.
19 . The computer program product of claim 18 , wherein the instructions further cause the processor to:
display information describing the item of the set in the position that is below positions in the ranking of each item of the set having the confidence score in the range of confidence scores indicating the moderate predicted availability and displaying the indication the item of the set has the low predicted availability in conjunction with the information describing the item of the set in the position that is below positions in the ranking of each item of the set having the confidence score in the range of confidence scores indicating the moderate predicted availability.
20 . The computer program product of claim 18 , the instructions, when executed by the processor, further cause the processor to:
responsive to the determined range of confidence scores indicating a low predicted availability of the item of the set, remove the item of the set from the ranking.
21 . The computer program product of claim 15 , wherein the instructions to decrease the position in the ranking of the item of the set further cause the processor to:
determine that the position in the ranking of the item of the set equals or exceeds a threshold position; and
in response to determining that the position in the ranking of the item of the set equals or exceeds a threshold position, maintain the position in the ranking of the item of the set.