Granted Patent B2

US 12,639,740 · App. 18/428,205 · Granted May 26, 2026

Inventors: Shreyas Saiprasad Jadhav (San Francisco, CA); Yue Xu (San Francisco, CA); Hyun Duk Cho (San Francisco, CA); Sushant Kumar (San Jose, CA); Kannan Achan (Saratoga, CA)

Assignee: Walmart Apollo, LLC

G06Q30/0631G06N5/022G06Q30/0641

Abstract

Claims (102)

1 . A system, comprising:

a processor; and

a non-transitory memory storing instructions that, when executed, cause the processor to:

generate training data based on historical transaction data and historical user session data of a plurality of users,

train at least one machine learning model during a training stage using the training data to optimize hyperparameters of the at least one machine learning model,

receive, from a computing device, a recommendation request for recommending items to a user,

determine, based on the recommendation request, an anchor item to be displayed to the user via a user interface,

determine relevance scores between the anchor item and a plurality of candidate items,

determine retention scores between the anchor item and the plurality of candidate items,

generate, using the at least one machine learning model, a ranked list of recommended items based at least by:

ranking the plurality of candidate items based on their respective relevance scores regarding the anchor item to generate a first ranking list of relevant items,

for each relevant item in the first ranking list, generating a weighted combination of a relevance score and a retention score regarding the anchor item using the at least one machine learning model based on a first weight for the relevance score and a second weight for the retention score, wherein the optimized hyperparameters of the at least one machine learning model include at least the first weight for the relevance score and the second weight for the retention score,

re-ranking the first ranking list to generate a second ranking list of relevant items based on the weighted combinations for the relevant items, and

selecting a plurality of top items in the second ranking list to generate the ranked list of recommended items, and

transmit to the computing device the ranked list of recommended items to be displayed to the customer user with the anchor item on the user interface.

2 . The system of claim 1 , wherein:

the user interface is associated with a website or an app of a retailer; and

the user is a new user who has never made any transaction with the retailer, or a reactivated user who has not made any transaction with the retailer during a past time period.

3 . The system of claim 1 , wherein the relevance scores are determined based on:

for each candidate item of the plurality of candidate items:

computing a similarity score between the anchor item and the candidate item;

computing a complementary score between the anchor item and the candidate item; and

computing a relevance score between the anchor item and the candidate item based on: the similarity score, the complementary score, or a combination of both.

4 . The system of claim 1 , wherein the retention scores are determined based on:

for each candidate item of the plurality of candidate items, computing a retention score indicating a probability that the user will purchase the candidate item within a future time period.

5 . The system of claim 4 , wherein computing the retention score comprises:

determining a first product type of the anchor item;

determining a second product type of the candidate item;

computing, based on a Bayesian statistical model, a retention probability that the user will purchase an item in the second product type within a certain time period after purchasing an item in the first product type; and

computing the retention score based on the retention probability.

6 . The system of claim 5 , wherein computing the retention probability comprises:

determining a first number of users who purchased items in the first product type;

determining, among the first number of users, a second number of users who purchased items in the second product type within the certain time period after purchasing items in the first product type; and

computing the retention probability based on a ratio between the second number and the first number.

7 . The system of claim 1 , wherein:

the training data includes a plurality of item pairs each including an anchor sample and a recommendation sample and associated with a label score;

the label score is determined based on interaction data of the plurality of users regarding the recommendation sample;

the at least one machine learning model is trained to optimize an objective function based on the optimized hyperparameters;

the objective function is computed based on a plurality of ranking differences each being a difference between a first ranking and a second ranking for a respective anchor sample;

each of the first ranking and the second ranking is a ranking of the recommendation samples paired to the respective anchor sample;

the first ranking of the recommendation samples is determined based on the at least one machine learning model; and

the second ranking of the recommendation samples is determined based on their respective label scores.

8 . The system of claim 7 , wherein:

the label score for each respective item pair is determined based on a combination of interaction scores representing different types of interactions of the plurality of users regarding a corresponding recommendation sample in the respective item pair;

the interaction scores include at least: a first interaction score representing an add-to-cart operation of a first user regarding the corresponding recommendation sample, a second interaction score representing a click operation of a second user regarding the corresponding recommendation sample, and a third interaction score representing an impression operation of a third user regarding the corresponding recommendation sample;

the first interaction score is larger than the second interaction score; and

the second interaction score is larger than the third interaction score.

9 . A computer-implemented method, comprising:

generating training data based on historical transaction data and historical user session data of a plurality of users;

training at least one machine learning model during a training stage using the training data to optimize hyperparameters of the at least one machine learning model;

receiving, from a computing device, a recommendation request for recommending items to a user;

determining, based on the recommendation request, an anchor item to be displayed to the user via a user interface;

determining relevance scores between the anchor item and a plurality of candidate items;

determining retention scores between the anchor item and the plurality of candidate items;

generating, using the at least one machine learning model, a ranked list of recommended items based at least by:

ranking the plurality of candidate items based on their respective relevance scores regarding the anchor item to generate a first ranking list of relevant items,

for each relevant item in the first ranking list, generating a weighted combination of a relevance score and a retention score regarding the anchor item using the at least one machine learning model based on a first weight for the relevance score and a second weight for the retention score, wherein the optimized hyperparameters of the at least one machine learning model include at least the first weight for the relevance score and the second weight for the retention score,

re-ranking the first ranking list to generate a second ranking list of relevant items based on the weighted combinations for the relevant items, and

selecting a plurality of top items in the second ranking list to generate the ranked list of recommended items; and

transmitting to the computing device the ranked list of recommended items to be displayed to the user with the anchor item on the user interface.

10 . The computer-implemented method of claim 9 , wherein determining the relevance scores comprises:

for each candidate item of the plurality of candidate items:

computing a similarity score between the anchor item and the candidate item;

computing a complementary score between the anchor item and the candidate item; and

computing a relevance score between the anchor item and the candidate item based on: the similarity score, the complementary score, or a combination of both.

11 . The computer-implemented method of claim 9 , wherein determining the retention scores comprises:

for each candidate item of the plurality of candidate items, computing a retention score indicating a probability that the customer user will purchase the candidate item within a future time period.

12 . The computer-implemented method of claim 11 , wherein computing the retention score comprises:

determining a first product type of the anchor item;

determining a second product type of the candidate item;

computing, based on a Bayesian statistical model, a retention probability that the user will purchase an item in the second product type within a certain time period after purchasing an item in the first product type; and

computing the retention score based on the retention probability.

13 . The computer-implemented method of claim 12 , wherein computing the retention probability comprises:

determining a first number of users who purchased items in the first product type;

determining, among the first number of users, a second number of users who purchased items in the second product type within the certain time period after purchasing items in the first product type; and

computing the retention probability based on a ratio between the second number and the first number.

14 . The computer-implemented method of claim 9 , wherein:

the training data includes a plurality of item pairs each including an anchor sample and a recommendation sample and associated with a label score;

the label score is determined based on interaction data of the plurality of users regarding the recommendation sample;

the at least one machine learning model is trained to optimize an objective function based on the optimized hyperparameters;

the objective function is computed based on a plurality of ranking differences each being a difference between a first ranking and a second ranking for a respective anchor sample;

each of the first ranking and the second ranking is a ranking of the recommendation samples paired to the respective anchor sample;

the first ranking of the recommendation samples is determined based on the at least one machine learning model; and

the second ranking of the recommendation samples is determined based on their respective label scores.

15 . The computer-implemented method of claim 14 , wherein:

the label score for each respective item pair is determined based on a combination of interaction scores representing different types of interactions of the plurality of users regarding a corresponding recommendation sample in the respective item pair;

the interaction scores include at least: a first interaction score representing an add-to-cart operation of a first user regarding the corresponding recommendation sample, a second interaction score representing a click operation of a second user regarding the corresponding recommendation sample, and a third interaction score representing an impression operation of a third user regarding the corresponding recommendation sample;

the first interaction score is larger than the second interaction score; and

the second interaction score is larger than the third interaction score.

16 . A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause at least one device to perform operations comprising:

generating training data based on historical transaction data and historical user session data of a plurality of users;

training at least one machine learning model during a training stage using the training data to optimize hyperparameters of the at least one machine learning model;

receiving, from a computing device, a recommendation request for recommending items to a user;

determining, based on the recommendation request, an anchor item to be displayed to the user via a user interface;

determining relevance scores between the anchor item and a plurality of candidate items;

determining retention scores between the anchor item and the plurality of candidate items;

generating, using the at least one machine learning model, a ranked list of recommended items based at least by:

ranking the plurality of candidate items based on their respective relevance scores regarding the anchor item to generate a first ranking list of relevant items,

for each relevant item in the first ranking list, generating a weighted combination of a relevance score and a retention score regarding the anchor item using the at least one machine learning model based on a first weight for the relevance score and a second weight for the retention score, wherein the optimized hyperparameters of the at least one machine learning model include at least the first weight for the relevance score and the second weight for the retention score,

re-ranking the first ranking list to generate a second ranking list of relevant items based on the weighted combinations for the relevant items, and

selecting a plurality of top items in the second ranking list to generate the ranked list of recommended items; and

transmitting to the computing device the ranked list of recommended items to be displayed to the user with the anchor item on the user interface.

Assignments (1)

Continuity (1)

References Cited (10)