SYSTEMS AND METHODS FOR GENERATING MODELS FOR RECOMMENDING REPLACEMENT ITEMS FOR UNAVAILABLE IN-STORE PURCHASES
In a “buy online, pick up in-store” service, customers place orders for items that are retrieved from store inventory and packaged for easy pick-up by the customer. Since these services typically fulfill orders from current store inventory, some items purchased by users are unavailable at the time of order fulfillment. In these circumstances, a recommendation system identifies a recommended replacement item using a trained model. The trained model includes a hierarchy of multiple sub-models, where each sub-model is configured to receive a different set of features of items as input and to generate, as output, a candidate recommended replacement item. A recommended replacement item is selected from the candidate recommendations generated by the multiple sub-models and sent for display to a user. The recommendation system receives user feedback regarding the recommended replacement item and selectively retrains one or more sub-models of the multiple sub-models based on the user feedback.
1 . A computer-implemented method comprising:
identifying, by a recommendation engine, a first item, of a set of one or more items purchased by a user for in-store fulfillment, that is unfulfillable at a time of intended fulfillment;
applying, by the recommendation engine, a trained model to select, from a set of additional items, a recommended replacement item for the first item,
wherein the trained model includes a hierarchy of multiple sub-models, each of the sub-models configured to receive a different set of features of additional items as input and to generate, as output, a candidate recommended replacement item; and
wherein the trained model is configured to select the recommended replacement item from the candidate recommended replacement items generated by the multiple sub-models;
sending, to a user device, information about the recommended replacement item for display to the user;
receiving, from an input on the user device, user feedback regarding the recommended replacement item; and
selectively retraining, by the recommendation engine, one or more sub-models of the multiple sub-models based on the user feedback.
2 . The method of claim 1 , wherein receiving the user feedback regarding the recommended replacement item comprises receiving a selection from the user of a different item instead of the recommended replacement item.
3 . The method of claim 2 , wherein selectively retraining the one or more sub-models based on the user feedback comprises:
identifying one or more features of the different item selected by the user;
identifying a sub-model of the multiple sub-models that is configured to receive, as input, a type of feature corresponding to the one or more identified features of the different item selected by the user; and
adding the one or more identified features of the different item selected by the user to a retraining data set for the identified sub-model.
4 . The method of claim 1 , wherein a first one of the sub-models is a gift categorization sub-model configured to output a prediction indicating whether the first item is likely to be a gift, and wherein a second one of the sub-models is a replacement gift selection sub-model configured to select a candidate replacement gift when the output of the gift categorization sub-model is a prediction that the first item is likely to be a gift.
5 . The method of claim 4 , wherein selectively retraining the one or more sub-models comprises retraining the replacement gift selection sub-model based on the user feedback when the output of the gift categorization sub-model is the prediction that the first item is likely to be a gift.
6 . The method of claim 1 , wherein the user feedback comprises an acceptance by the user of the recommended replacement item, and wherein selectively retraining the one or more sub-models based on the user feedback comprises adding the recommended replacement item to a positive training set for retraining the one or more sub-models.
7 . The method of claim 1 , wherein the user feedback comprises a cancellation of an order for the first item, and wherein selectively retraining the one or more sub-models based on the user feedback comprises adding the recommended replacement item to a negative training set for retraining the one or more sub-models.
8 . The method of claim 1 , wherein selecting the recommended replacement item from the candidate recommended replacement items comprises:
applying a ranking to the candidate recommended replacement items; and
selecting the recommended replacement item based on the ranking.
9 . The method of claim 8 , wherein applying the ranking comprises:
determining an expected likelihood of availability of each of the candidate recommended replacement items at the time of intended fulfillment; and
ranking the candidate replacement items based on the expected likelihood of availability.
10 . The method of claim 1 , wherein the trained model further comprises a recommendation selection model trained to select the recommended replacement item from among the candidate recommended replacement items, and wherein selectively retraining one or more of the sub-models comprises:
in response to determining the user selected a different item from the candidate recommended replacement items instead of the recommended replacement item, adding information about the different item selected from the candidate recommended replacement items to a retraining data set for the recommendation selection model; and
retraining the recommendation selection model based on the retraining data set.
11 . The method of claim 1 , further comprising:
determining the first item is unfulfillable based on a notification received from a store inventory system.
12 . The method of claim 1 , further comprising:
querying a store inventory system at a specified time prior to the time of intended fulfillment; and
determining the first item is unfulfillable based on a response to the query.
13 . The method of claim 1 , further comprising:
receiving input from a store employee indicating the first item is unfulfillable.
14 . The method of claim 1 , wherein applying the trained model comprises:
identifying a set of items within a store inventory having a likelihood of availability at the time of intended fulfillment that is greater than a threshold;
applying one or more of the sub-models to the identified set of items to generate respective candidate recommended replacement items.
15 . The method of claim 1 , wherein applying the trained model comprises:
determining, for each of the candidate recommended replacement items, a likelihood of availability of the respective candidate recommended replacement item at the time of intended fulfillment; and
selecting the recommended replacement item from the candidate recommended replacement items based on the respective likelihoods of availability.
16 . A non-transitory computer-readable storage medium storing executable computer program instructions, the computer program instructions when executed by a processor causing the processor to:
identify a first item, of a set of one or more items purchased by a user for in-store fulfillment, that is unfulfillable at a time of intended fulfillment;
apply a trained model to select, from a set of additional items, a recommended replacement item for the first item,
wherein the trained model includes a hierarchy of multiple sub-models, each of the sub-models configured to receive a different set of features of additional items as input and generate, as output, a candidate recommended replacement item; and
wherein the trained model is configured to select the recommended replacement item from the candidate recommended replacement items generated by the multiple sub-models;
send, to a user device, information about the recommended replacement item for display to the user;
receive, via an input on the user device, user feedback regarding the recommended replacement item; and
selectively retrain one or more sub-models of the multiple sub-models based on the user feedback.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein receiving the user feedback regarding the recommended replacement item comprises receiving a selection from the user of a different item instead of the recommended replacement item, and wherein selectively retraining the one or more sub-models based on the user feedback comprises:
identifying one or more features of the different item selected by the user;
identifying a sub-model of the multiple sub-models that is configured to receive, as input, a type of feature corresponding to the one or more identified features of the different item selected by the user; and
adding the one or more identified features of the different item selected by the user to a retraining data set for the identified sub-model.
18 . The non-transitory computer-readable storage medium of claim 16 , wherein applying the trained model comprises:
identifying a set of items within a store inventory having a likelihood of availability at the time of intended fulfillment that is greater than a threshold;
applying one or more of the sub-models to the identified set of items to generate respective candidate recommended replacement items.
19 . The non-transitory computer-readable storage medium of claim 16 , wherein applying the trained model comprises:
determining, for each of the candidate recommended replacement items, a likelihood of availability of the respective candidate recommended replacement item at the time of intended fulfillment; and
selecting the recommended replacement item from the candidate recommended replacement items based on the respective likelihoods of availability.
20 . A replacement product recommendation system, comprising:
one or more processors; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the one or more processors, cause the one or more processors to:
identify a first item, of a set of one or more items purchased by a user for in-store fulfillment, that is unfulfillable at a time of intended fulfillment;
apply a trained model to select, from a set of additional items, a recommended replacement item for the first item,
wherein the trained model includes a hierarchy of multiple sub-models, each of the sub-models configured to receive a different set of features of additional items as input and generate, as output, a candidate recommended replacement item; and
wherein the trained model is configured to select the recommended replacement item from the candidate recommended replacement items generated by the multiple sub-models;
send information about the recommended replacement item for display to the user;
receive user feedback regarding the recommended replacement item; and
selectively retrain one or more sub-models of the multiple sub-models based on the user feedback.