Identifying candidate replacement items with a source similarity score
An online concierge system suggests replacement items when an ordered item may be unavailable. To promote similarity of sources between the replacement item with the ordered item, candidate replacement items are scored, in part, based on a source similarity score based on a source of the candidate replacement item and a source of the ordered item. The source similarity score may be determined by a computer model based on user interactions with item sources. The similarity score may be based on source embeddings that may be determined based on respective item embeddings or may be determined by training source embeddings directly from user-source interactions. The similarity score for a candidate replacement item may be combined with a replacement score indicating the user's likelihood of selecting the candidate replacement item as a replacement to yield a total score for selection as suggestion as a replacement for the ordered item.
1. A method performed by one or more computer processors, the method comprising:
receiving an order comprising a set of items for a user;
identifying an ordered item from the set of items for suggesting a replacement item to the user;
determining a set of candidate replacement items for the ordered item;
determining a total score for each candidate replacement item in the set of candidate replacement items by:
determining a replacement score for the candidate replacement item indicating a likelihood that the candidate replacement item would be selected by the user as a replacement for the ordered item,
determining a source similarity score indicating a similarity between a first source of the ordered item and a second source of the candidate replacement item, wherein determining the source similarity score comprises:
accessing a first source embedding for the first source, wherein the first source embedding is generated by source embedding layers of a machine-learning model, wherein the machine-learning model comprises the source embedding layers and user embedding layers and is trained to predict user-source interactions based on user data and source feature data associated with a source, wherein the machine-learning model is trained based on training examples comprising source features associated with a source for input to the source embedding layers, user data describing a user for input to the user embedding layers, and a label indicating whether the user interacted with the source,
accessing a second source embedding for the second source, wherein the second source embedding is generated by the source embedding layers of the machine-learning model, and
computing a difference between the first source embedding and the second source embedding, and
combining the replacement score and the source similarity score to generate the total score;
selecting one or more candidate replacement items based on the total scores; and
transmitting information identifying the one or more selected replacement items for display to the user.
2. The method of claim 1 , wherein identifying the ordered item from the set of items for suggesting a replacement item to the user comprises:
applying an item availability model that is trained to predict a likelihood that the ordered item is available at a location for fulfilling the order, the item availability model trained based on training data that describes historical fulfillment of previous orders and availability of items in the previous orders; and
declaring the ordered item to be unavailable based on the predicted likelihood that the ordered item is available.
3. The method of claim 1 , wherein determining the replacement score for each candidate replacement item comprises:
applying a trained replacement model to a set of features describing the candidate replacement item, the trained replacement model configured to output a likelihood that the candidate replacement item would be selected by the user as a replacement for the ordered item,
wherein the replacement model is trained based on training data that describes historical suggestions of replacement items to other users and previous user responses thereto.
4. The method of claim 1 , wherein determining the source similarity score comprises:
generating the first source embedding associated with the first source based on one or more item embeddings of products associated with the first source; and
generating the second source embedding associated with the second source based on one or more item embeddings of products associated with the second source.
5. The method of claim 1 , wherein combining the replacement score and the source similarity score to generate the total score comprises computing a weighted sum of the replacement score and the source similarity score.
6. The method of claim 1 , wherein combining the replacement score and the source similarity score to generate the total score comprises applying a function that decreases as the source similarity score decreases.
7. The method of claim 1 , wherein determining a total score for each candidate replacement item further comprises adjusting the total score by a bid value associated with the respective candidate replacement item.
8. The method of claim 7 , wherein adjusting the total score by a bid value comprises multiplying the bid value by the similarity score.
9. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:
receiving an order comprising a set of items for a user;
identifying an ordered item from the set of items for suggesting a replacement item to the user;
determining a set of candidate replacement items for the ordered item;
determining a total score for each candidate replacement item in the set of candidate replacement items by:
determining a replacement score for the candidate replacement item indicating a likelihood that the candidate replacement item would be selected by the user as a replacement for the ordered item,
determining a source similarity score indicating a similarity between a first source of the ordered item and a second source of the candidate replacement item, wherein determining the source similarity score comprises:
accessing a first source embedding for the first source, wherein the first source embedding is generated by source embedding layers of a machine-learning model, wherein the machine-learning model comprises the source embedding layers and user embedding layers and is trained to predict user-source interactions based on user data and source feature data associated with a source, wherein the machine-learning model is trained based on training examples comprising source features associated with a source for input to the source embedding layers, user data describing a user for input to the user embedding layers, and a label indicating whether the user interacted with the source,
accessing a second source embedding for the second source, wherein the second source embedding is generated by the source embedding layers of the machine-learning model, and
computing a difference between the first source embedding and the second source embedding, and
combining the replacement score and the source similarity score to generate the total score;
selecting one or more candidate replacement items based on the total scores; and
transmitting information identifying the one or more selected replacement items for display to the user.
10. The computer program product of claim 9 , wherein identifying the ordered item from the set of items for suggesting a replacement item to the user comprises:
applying an item availability model that is trained to predict a likelihood that the ordered item is available at a location for fulfilling the order, the item availability model trained based on training data that describes historical fulfillment of previous orders and availability of items in the previous orders; and
declaring the ordered item to be unavailable based on the predicted likelihood that the ordered item is available.
11. The computer program product of claim 9 , wherein determining the replacement score for each candidate replacement item comprises:
applying a trained replacement model to a set of features describing the candidate replacement item, the trained replacement model configured to output a likelihood that the candidate replacement item would be selected by the user as a replacement for the ordered item,
wherein the replacement model is trained based on training data that describes historical suggestions of replacement items to other users and previous user responses thereto.
12. The computer program product of claim 9 , wherein determining the source similarity score comprises:
generating the first source embedding associated with the first source based on one or more item embeddings of products associated with the first source; and
generating the second source embedding associated with the second source based on one or more item embeddings of products associated with the second source.
13. The computer program product of claim 12 , wherein the first source embedding and the second source embedding are jointly trained with a set of user embeddings based on a training set of user-source interactions of users interacting with items having associated sources.
14. The computer program product of claim 9 , wherein combining the replacement score and the source similarity score to generate the total score comprises applying a function that decreases as the source similarity score decreases.
15. The computer program product of claim 9 , wherein determining a total score for each candidate replacement item further comprises adjusting the total score by a bid value associated with the respective candidate replacement item.
16. The computer program product of claim 15 , wherein adjusting the total score by a bid value comprises multiplying the bid value by the similarity score.
17. A system comprising:
one or more processors; and
a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the one or more processors, cause the one or more processors to perform steps comprising:
receiving an order comprising a set of items for a user;
identifying an ordered item from the set of items for suggesting a replacement item to the user;
determining a set of candidate replacement items for the ordered item;
determining a total score for each candidate replacement item in the set of candidate replacement items by:
determining a replacement score for the candidate replacement item indicating a likelihood that the candidate replacement item would be selected by the user as a replacement for the ordered item,
determining a source similarity score indicating a similarity between a first source of the ordered item and a second source of the candidate replacement item, wherein determining the source similarity score comprises:
accessing a first source embedding for the first source, wherein the first source embedding is generated by source embedding layers of a machine-learning model, wherein the machine-learning model comprises the source embedding layers and user embedding layers and is trained to predict user-source interactions based on user data and source feature data associated with a source, wherein the machine-learning model is trained based on training examples comprising source features associated with a source for input to the source embedding layers, user data describing a user for input to the user embedding layers, and a label indicating whether the user interacted with the source,
accessing a second source embedding for the second source, wherein the second source embedding is generated by the source embedding layers of the machine-learning model, and
computing a difference between the first source embedding and the second source embedding, and
combining the replacement score and the source similarity score to generate the total score;
selecting one or more candidate replacement items based on the total scores; and
transmitting information identifying the one or more selected replacement items for display to the user.
18. The system of claim 17 , wherein identifying the ordered item from the set of items for suggesting a replacement item to the user comprises:
applying an item availability model that is trained to predict a likelihood that the ordered item is available at a location for fulfilling the order, the item availability model trained based on training data that describes historical fulfillment of previous orders and availability of items in the previous orders; and
declaring the ordered item to be unavailable based on the predicted likelihood that the ordered item is available.