IP Library › Granted Patent US 11,354,719
Granted Patent B2
US 11,354,719 · App. 16/287,740 · Granted Jun 7, 2022

Systems and methods for determining substitutions

Inventors: Kamiya Motwani (Sunnyvale, CA); Sushant Kumar (Sunnyvale, CA); Kannan Achan (Saratoga, CA); Vidya Sagar Kalidindi (Milpitas, CA); Rahul Ramkumar (Santa Clara, CA); Derrick Lagomarsino (San Mateo, CA)
Assignee: WALMART APOLLO, LLC
G06Q30/0631G06Q30/0627G06Q30/0629
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,354,719
App. No.
16/287,740
Granted
Jun 7, 2022
Kind
B2
Abstract

Systems and methods including one or more processors and one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and perform: determining a list of substitutes for an item; determining qualities for each substitute; determining a similarity score for each substitute; determining a historical substitution score for each substitute; determining a final score for each substitute using the similarity score for each substitute and the historical substitution score for each substitute; ranking each substitute based upon the final score for each substitute; facilitating a display, on a user interface of a user device, of a highest ranked substitute; receiving, from the user interface of the user device, a selection of the highest ranked substitute; and after receiving the selection of the highest ranked substitute, substituting the highest ranked substitute for the item of the list of items. Other embodiments are disclosed herein.

Claims (126)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable storage devices storing computing instructions configured to run on the one or more processors and cause the processors to perform:

determining when an item of a list of items is out of stock; and

when the item is out of stock:

determining a list of possible substitutes for the item of the list of items, the list of possible substitutes comprising a set of items in an item catalogue;

determining respective qualities for each possible substitute of the list of possible substitutes;

training a machine learning algorithm on labeled training data comprising (1) all positive historical acceptance data and (2) randomly sampled negative historical acceptance data, wherein each labeled training datum of the labeled training data is labeled with at least one of:

a respective probability of a respective similarity between a respective historical title for a respective historical item and a respective historical description of a respective historical substitute;

a respective taxonomy difference for the respective historical item and the respective historical substitute;

a respective price difference between the respective historical item and the respective historical substitute; or

a respective normalized sales rank comparison for the respective historical item and the respective historical substitute;

determining a respective similarity score for each possible substitute of the list of possible substitutes using the machine learning algorithm, as trained;

determining a respective historical substitution score for each possible substitute of the list of possible substitutes;

determining a respective final score for each possible substitute of the list of possible substitutes using the respective similarity score for each possible substitute of the list of possible substitutes, the respective historical substitution score for each possible substitute of the list of possible substitutes, and at least one or more rectifiers having ReLU non-linearity that enables training of deep supervised neural networks without unsupervised pre-training;

ranking each possible substitute of the list of possible substitutes based upon the respective final score for each possible substitute;

facilitating a display, on a user interface of a user device, of a highest ranked possible substitute of the list of possible substitutes;

receiving, from the user interface of the user device, a selection of the highest ranked possible substitute of the list of possible substitutes;

after receiving the selection of the highest ranked possible substitute, substituting the highest ranked possible substitute of the list of possible substitutes for the item of the list of items;

storing the selection of the highest ranked possible substitute as additional training data with the labeled training data; and

re-training the machine learning algorithm on the additional training data and the labeled training data.

2. The system of claim 1 , wherein determining the list of possible substitutes comprises:

accessing an item taxonomy database comprising a respective item taxonomy for each item in a catalogue of items, the catalogue of items comprising the item of the list of items;

identifying a specific item taxonomy for the item of the list of items;

filtering out non-matching items of the catalogue of items, the non-matching items having a different item taxonomy than the specific item taxonomy of the item of the list of items; and

adding matching items of the catalogue of items to the list of possible substitutes, the matching items of the catalogue of items having the specific item taxonomy.

3. The system of claim 1 , wherein:

the respective qualities for each possible substitute of the list of possible substitutes comprise:

a respective brand for each possible substitute of the list of possible substitutes;

a respective attribute for each possible substitute of the list of possible substitutes;

a respective product for each possible substitute of the list of possible substitutes; and

a respective dietary restriction for each possible substitute of the list of possible substitutes; and

when the item is out of stock, the computing instructions are further configured to run on the one or more processors and cause the one or more processors to perform:

receiving a respective title for each possible substitute of the list of possible substitutes;

identifying the respective brand for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes;

identifying the respective attribute for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes;

identifying the respective product for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes; and

identifying the respective dietary restriction for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes.

4. The system of claim 3 , wherein determining the respective similarity score for each possible substitute of the list of possible substitutes comprises:

determining a respective title similarity score for each possible substitute of the list of possible substitutes by comparing:

the respective brand for each possible substitute of the list of possible substitutes and a brand of the item of the list of items;

the respective attribute for each possible substitute of the list of possible substitutes and an attribute of the item of the list of items; and

the respective product for each possible substitute of the list of possible substitutes and a product of the item of the list of items.

5. The system of claim 4 , wherein determining the respective title similarity score for each possible substitute of the list of possible substitutes comprises using a logistic regressor trained on:

respective historical acceptance data for each possible substitute of the list of possible substitutes; and

manually chosen substitute data.

6. The system of claim 3 , wherein, when the item is out of stock, the computing instructions are further configured to run on the one or more processors and cause the one or more processors to perform:

comparing the respective dietary restriction for each possible substitute of the list of possible substitutes with a dietary restriction of the item of the list of items; and

when a dietary restriction of a possible substitute of the list of possible substitutes does not match the dietary restriction of the item of the list of items, removing the possible substitute of the list of possible substitutes from the list of possible substitutes.

7. The system of claim 3 , wherein identifying the respective brand, the respective attribute, the respective product, and the respective dietary restriction comprises using a Hidden Markov Model.

8. The system of claim 1 , wherein ranking each possible substitute of the list of possible substitutes comprises:

when two or more possible substitutes have approximately similar final scores, determining a quantity ratio comprising a ratio of a quantity for the item of the list of items to a respective quantity for each possible substitute of the list of possible substitutes, wherein:

the quantity for the item of the list of items comprises:

either (1) a weight of the item of the list of items or (2) a volume of the item of the list of items, divided by a count of the item of the list of items; and

the respective quantity for each possible substitute of the list of possible substitutes comprises:

either (1) a respective weight of each possible substitute of the list of possible substitutes or (2) a respective volume of each possible substitute of the list of possible substitutes, divided by a respective count of each possible substitute of the list of possible substitutes; and

ranking a possible substitute of the list of possible substitutes with a lower quantity ratio above a possible substitute of the list of possible substitutes with a higher quantity ratio.

9. The system of claim 1 , wherein determining the respective historical substitution score comprises:

determining a respective number of successful substitutions for each possible substitute of the list of possible substitutes;

determining a respective number of unsuccessful substitutions for each possible substitute of the list of possible substitutes;

calculating a respective historical acceptance rate for each possible substitute of the list of possible substitutes using the respective number of successful substitutions for each possible substitute of the list of possible substitutes and the respective number of unsuccessful substitutions for each possible substitute of the list of possible substitutes; and

assigning the respective historical substitution score based on the respective historical acceptance rate for each substitution of the list of possible substitutes and the respective number of successful substitutions for each possible substitute of the list of possible substitutes.

10. The system of claim 1 , wherein determining the respective final score comprises:

inputting the respective similarity score for each possible substitute of the list of possible substitutes and the respective historical substitution score for each possible substitute of the list of possible substitutes into a feed-forward neural network comprising the one or more rectifiers having ReLU non-linearity, wherein the feed-forward neural network is trained without unsupervised pre-training.

11. A method being implemented via execution of computing instructions configured to run at one or more processors and configured to be stored at non-transitory computer-readable media, the method comprising:

determining when an item of a list of items is out of stock; and

when the item is out of stock:

determining a list of possible substitutes for the item of the list of items, the list of possible substitutes comprising a set of items in an item catalogue;

determining respective qualities for each possible substitute of the list of possible substitutes;

training a machine learning algorithm on labeled training data comprising (1) all positive historical acceptance data and (2) randomly sampled negative historical acceptance data, wherein each labeled training datum of the labeled training data is labeled with at least one of:

a respective probability of a respective similarity between a respective historical title for a respective historical item and a respective historical description of a respective historical substitute;

a respective taxonomy difference for the respective historical item and the respective historical substitute;

a respective price difference between the respective historical item and the respective historical substitute; or

a respective normalized sales rank comparison for the respective historical item and the respective historical substitute;

determining a respective similarity score for each possible substitute of the list of possible substitutes using the machine learning algorithm, as trained;

determining a respective historical substitution score for each possible substitute of the list of possible substitutes;

determining a respective final score for each possible substitute of the list of possible substitutes using the respective similarity score for each possible substitute of the list of possible substitutes, the respective historical substitution score for each possible substitute of the list of possible substitutes, and at least one or more rectifiers having ReLU non-linearity that enables training of deep supervised neural networks without unsupervised pre-training;

ranking each possible substitute of the list of possible substitutes based upon the respective final score for each possible substitute;

facilitating a display, on a user interface of a user device, of a highest ranked possible substitute of the list of possible substitutes;

receiving, from the user interface of the user device, a selection of the highest ranked possible substitute of the list of possible substitutes;

after receiving the selection of the highest ranked possible substitute, substituting the highest ranked possible substitute of the list of possible substitutes for the item of the list of items;

storing the selection of the highest ranked possible substitute as additional training data with the labeled training data; and

re-training the machine learning algorithm on the additional training data and the labeled training data.

12. The method of claim 11 , wherein determining the list of possible substitutes comprises:

accessing an item taxonomy database comprising a respective item taxonomy for each item in a catalogue of items, the catalogue of items comprising the item of the list of items;

identifying a specific item taxonomy for the item of the list of items;

filtering out non-matching items of the catalogue of items, the non-matching items having a different item taxonomy than the specific item taxonomy of the item of the list of items; and

adding matching items of the catalogue of items to the list of possible substitutes, the matching items of the catalogue of items having the specific item taxonomy.

13. The method of claim 11 , wherein:

the respective qualities for each possible substitute of the list of possible substitutes comprise:

a respective brand for each possible substitute of the list of possible substitutes;

a respective attribute for each possible substitute of the list of possible substitutes;

a respective product for each possible substitute of the list of possible substitutes; and

a respective dietary restriction for each possible substitute of the list of possible substitutes; and

when the item is out of stock, the method further comprises:

receiving a respective title for each possible substitute of the list of possible substitutes;

identifying the respective brand for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes;

identifying the respective attribute for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes;

identifying the respective product for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes; and

identifying the respective dietary restriction for each possible substitute of the list of possible substitutes using the respective title for each possible substitute of the list of possible substitutes.

14. The method of claim 13 , wherein determining the respective similarity score for each possible substitute of the list of possible substitutes comprises:

determining a respective title similarity score for each possible substitute of the list of possible substitutes by comparing:

the respective brand for each possible substitute of the list of possible substitutes and a brand of the item of the list of items;

the respective attribute for each possible substitute of the list of possible substitutes and an attribute of the item of the list of items; and

the respective product for each possible substitute of the list of possible substitutes and a product of the item of the list of items.

15. The method of claim 14 , wherein determining the respective title similarity score for each possible substitute of the list of possible substitutes comprises using a logistic regressor trained on:

respective historical acceptance data for each possible substitute of the list of possible substitutes; and

manually chosen substitute data.

16. The method of claim 13 , wherein, when the item is out of stock, the method further comprises:

comparing the respective dietary restriction for each possible substitute of the list of possible substitutes with a dietary restriction of the item of the list of items; and

when a dietary restriction of a possible substitute of the list of possible substitutes does not match the dietary restriction of the item of the list of items, removing the possible substitute of the list of possible substitutes from the list of possible substitutes.

17. The method of claim 13 , wherein identifying the respective brand, the respective attribute, the respective product, and the respective dietary restriction comprises using a Hidden Markov Model.

18. The method of claim 11 , wherein ranking each possible substitute of the list of possible substitutes comprises:

when two or more possible substitutes have approximately similar final scores, determining a quantity ratio comprising a ratio of a quantity for the item of the list of items to a respective quantity for each possible substitute of the list of possible substitutes, wherein:

the quantity for the item of the list of items comprises:

either (1) a weight of the item of the list of items or (2) a volume of the item of the list of items, divided by a count of the item of the list of items; and

the respective quantity for each possible substitute of the list of possible substitutes comprises:

either (1) a respective weight of each possible substitute of the list of possible substitutes or (2) a respective volume of each possible substitute of the list of possible substitutes, divided by a respective count of each possible substitute of the list of possible substitutes; and

ranking a possible substitute of the list of possible substitutes with a lower quantity ratio above a possible substitute of the list of possible substitutes with a higher quantity ratio.

19. The method of claim 11 , wherein determining the respective historical substitution score comprises:

determining a respective number of successful substitutions for each possible substitute of the list of possible substitutes;

determining a respective number of unsuccessful substitutions for each possible substitute of the list of possible substitutes;

calculating a respective historical acceptance rate for each possible substitute of the list of possible substitutes using the respective number of successful substitutions for each possible substitute of the list of possible substitutes and the respective number of unsuccessful substitutions for each possible substitute of the list of possible substitutes; and

assigning the respective historical substitution score based on the respective historical acceptance rate for each substitution of the list of possible substitutes and the respective number of successful substitutions for each possible substitute of the list of possible substitutes.

20. The method of claim 11 , wherein determining the respective final score comprises:

inputting the respective similarity score for each possible substitute of the list of possible substitutes and the respective historical substitution score for each possible substitute of the list of possible substitutes into a feed-forward neural network comprising the one or more rectifiers having ReLU non-linearity, wherein the feed-forward neural network is trained without unsupervised pre-training.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 2, 2019
From: MOTWANI, KAMIYA; KUMAR, SUSHANT; ACHAN, KANNAN; KALIDINDI, VIDYA SAGAR; RAMKUMAR, RAHUL; LAGOMARSINO, DERRICK
To: WALMART APOLLO, LLC
Reel/Frame 048774/0956 →
Continuity (1)
Related Publication 20200273083A1 · Aug 27, 2020
Cited By (1)
US 12,670,471