IP Library › Granted Patent US 12,749,477
Granted Patent B2
US 12,749,477 · App. 18/816,342 · Granted Sep 29, 2026

Systems and methods to identify products from verbal utterances

Inventors: Praneeth Gubbala (Milpitas, CA); Xuan Zhang (Georgetown, TX); Bahula Bosetti (Bentonville, AR); Priya Ashok Kumar Choudhary (Sunnyvale, CA); Dong T. Nguyen (Wylie, TX); Shivraj V. Kodak (San Mateo, CA); William Craig Robinson, Jr. (Centerton, AR)
Assignee: Walmart Apollo, LLC
G10L15/063G06F16/685G10L15/02G10L2015/086
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,749,477
App. No.
18/816,342
Granted
Sep 29, 2026
Kind
B2
Abstract

Some embodiments provide retail product ordering systems comprising: a user computing device comprising an application executed by a device control circuit to: receive an audible utterance; controls a product identifier application interface to: apply a tokenizer model and obtain a set of individual search words; apply a series of featurizer models to the search words to generate features; and apply a classifier and extractor model based on the features and generate multiple requested product entities each comprising a respective sub-set of the position labeled product terms; wherein the device control circuit is further configured to access a purchase history database, confirm an accuracy of each of requested product entities relative to a purchase history, generate a listing of determined product identifiers corresponding to the confirmed set of the multiple requested product entities, and control a display system of the user computing device to render the listing of determined product identifiers.

Claims (57)

1 . A retail product ordering system comprising:

a user computing device comprising a transceiver, an audio detecting system, and an application stored in memory that, when executed by a device control circuit of the user computing device, is configured to:

receive an audible utterance of a request comprising a series of request words;

obtain a plurality of search words corresponding to the series of request words by applying a tokenizer model to the series of request words, the tokenizer model trained based on training utterance words;

generate features of the plurality of search words by applying a series of featurizer models to the set of individual search words, the series of featurizer models trained based on sets of predefined product identifiers corresponding to a respective single product;

obtain a plurality of requested product entities based on the features of the plurality of search words by applying a classifier and extractor model, the classifier and extractor model trained based on the sets of predefined product identifiers comprising product terms, each product term having position labels, wherein each requested product entity of the plurality of requested product entities comprises a respective sub-set of position labeled product terms ordered in accordance with corresponding position labels; and

generate a recommendation listing comprising recommended products, the recommended products corresponding to the plurality of requested product entities.

2 . The retail product ordering system of claim 1 , wherein generating the recommendation listing comprises:

accessing a product inventory database;

for each of the plurality of requested product entities, identifying a category of recommended products in the product inventory database; and

generating the recommendation listing, wherein the recommendation listing comprises the categories of the recommended products.

3 . The retail product ordering system of claim 2 , wherein the product inventory database corresponds to one or more retail entities.

4 . The retail product ordering system of claim 1 , wherein generating the recommendation listing comprises:

accessing purchase history data associated with a plurality of users;

for each of the plurality of requested product entities, identifying a common actual product purchased by the plurality of users from the purchase history data; and

generating the recommendation listing, wherein the recommendation listing comprises the common actual products.

5 . The retail product ordering system of claim 4 , wherein the plurality of users have one or more threshold correlations with a user submitting the audible utterance.

6 . The retail product ordering system of claim 1 , wherein the device control circuit is further configured to:

control a display system of the user computing device to render the recommendation listing of the recommended products.

7 . The retail product ordering system of claim 6 , wherein the displayed recommendation listing of the recommended products comprises a selectable option that is configured to cause, in response to selection of the selectable option by a user, product information to be displayed for each product associated with the recommended products.

8 . A method of enabling retail product ordering comprising:

receiving an audible utterance of a request comprising a series of request words;

obtaining a plurality of search words corresponding to the series of request words by applying a tokenizer model to the series of request words, the tokenizer model trained based on training utterance words;

generating features of the plurality of search words by applying a series of featurizer models to the set of individual search words, the series of featurizer models trained based on set of predefined product identifiers corresponding to a respective single product;

obtaining a plurality of requested product entities based on the features of the plurality of search words by applying a classifier and extractor model, the classifier and extractor model trained based on the sets of predefined product identifiers comprising product terms, each product term having position labels, wherein each requested product entity of the plurality of requested product entities comprises a respective sub-set of position labeled product terms ordered in accordance with corresponding position labels; and

generating a recommendation listing comprising recommended products, the recommended products corresponding to the plurality of requested product entities.

9 . The method of claim 8 , wherein generating the recommendation listing comprises:

accessing a product inventory database;

for each of the plurality of requested product entities, identifying a category of recommended products in the product inventory database; and

generating the recommendation listing, wherein the recommendation listing comprises the categories of the recommended products.

10 . The method of claim 9 , wherein the product inventory database corresponds t one or more retail entities.

11 . The method of claim 8 , wherein generating the recommendation listing comprises:

accessing purchase history data associated with a plurality of users;

for each of the plurality of requested product entities, identifying a common actual product purchased by the plurality of users from the purchase history data; and

generating the recommendation listing, wherein the recommendation listing comprises the common actual products.

12 . The method of claim 11 , wherein the plurality of users have one or more threshold correlations with a user submitting the audible utterance.

13 . The method of claim 8 , further comprising:

displaying the recommendation listing of the recommended products.

14 . The method of claim 13 , wherein the displayed recommendation listing of the recommended products comprises a selectable option that is configured to cause, in response to selection of the selectable option by a user, product information to be displayed for each product associated with the recommended products.

15 . A non-transitory computer-readable media storing a plurality of instructions, which, when executed by a processor, cause the processor to perform operations comprising:

receiving an audible utterance of a request comprising a series of request words;

obtaining a plurality of search words corresponding to the series of request words by applying a tokenizer model to the series of request words, the tokenizer model trained based on training utterance words;

generating features of the plurality of search words by applying a series of featurizer models to the set of individual search words, the series of featurizer models trained based on sets of predefined product identifiers corresponding to a respective single product;

obtaining a plurality of requested product entities based on the features of the plurality of search words by applying a classifier and extractor model, the classifier and extractor model trained based on the sets of predefined product identifiers comprising product terms, each product term having position labels, wherein each requested product entity of the plurality of requested product entities comprises a respective sub-set of position labeled product terms ordered in accordance with corresponding position labels; and

generating a recommendation listing comprising recommended products, the recommended products corresponding to the plurality of requested product entities.

16 . The non-transitory computer-readable media of claim 15 , wherein generating the recommendation listing comprises:

accessing a product inventory database;

for each of the plurality of requested product entities, identifying a category of recommended products in the product inventory database; and

generating the recommendation listing, wherein the recommendation listing comprises the categories of the recommended products.

17 . The non-transitory computer-readable media of claim 16 , wherein the product inventory database corresponds to one or more retail entities.

18 . The non-transitory computer-readable media of claim 15 , wherein the operations further comprise:

accessing purchase history data associated with a plurality of users, wherein the plurality of users have one or more threshold correlations with a user submitting the audible utterance;

for each of the plurality of requested product entities, identifying a common actual product purchased by the plurality of users from the purchase history data; and

generating the recommendation listing, wherein the recommendation listing comprises the common actual products.

19 . The non-transitory computer-readable media of claim 15 , wherein generating the recommendation listing comprises:

displaying the recommendation listing of the recommended products.

20 . The non-transitory computer-readable media of claim 19 , wherein the displayed recommendation listing of the recommended products comprises a selectable option that is configured to cause, in response to selection of the selectable option by a user, product information to be displayed for each product associated with the recommended products.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2024
From: GUBBALA, PRANEETH; ZHANG, XUAN; BOSETTI, BAHULA; CHOUDHARY, PRIYA ASHOK KUMAR; NGUYEN, DONG T.; KODAK, SHIVRAJ V.; ROBINSON, WILLIAM CRAIG, JR.
To: WALMART APOLLO, LLC
Reel/Frame 068987/0499 →
Continuity (2)
Continuation 17730027 · Apr 26, 2022
Related Publication 20240420683A1 · Dec 19, 2024
References Cited (21)
US 6829603B1 · Chai · 2004 [cited by applicant]
US 7937263B2 · Carrier · 2011 [cited by examiner]
US 8078502B2 · Hao · 2011 [cited by examiner]
US 10872601B1 · Acharya · 2020 [cited by examiner]
US 20200175566A1 · Bender · 2020 [cited by examiner]
US 20210248321A1 · Cheng · 2021 [cited by examiner]
CN 111444719 · 2020 [cited by applicant]
Agarwal et al, “Parsing Coordination for Spoken Language Understanding”; Oct. 26, 2018, IEEE 2018 Spoken Language Technology (SLT 2018), pp. 1-8. [cited by applicant]
Agarwal, Sanchit, “How Alexa Knows “Peanut Butter” is One Shopping-List Item, Not Two”, https ://www.amazon.science/blog/how-alexa-knows-peanut-butter-is-one-shopping-list-item-not-two, Dec. 18, 2018, 8 pages. [cited by applicant]
Bunk et al, “DIET: Lightweight Language Understanding for Dialogue Systems”, https//:doi.org/10.48550/arXiv.2004.09936, Apr. 21, 2020, 9 pages. [cited by applicant]
Devlin et al., “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding”, https://doi. org/10.48550/arXiv.1810.04805, Oct. 11, 2018, 16 pages. [cited by applicant]
Gubbala et al., “Sequence to Sequence Model for Extracting Multiple Product NameEntities”; https://www.youtube.com/watch?v=WPzTOA1tbfw, Oct. 29, 2021, pp. 1. [cited by applicant]
Gubbala et al., “A Sequence to Sequence Model for Extracting Multiple Product Name Entities from Dialog”, WeCNLP 2021, Oct. 29, 2021, 3 pages. [cited by applicant]
Henderson et al, “ConveRT: Efficient and Accurate Conversational Representations from Transformers”; https://doi.org/10.48550/arXiv.1911.03688; Nov. 9, 2019; 11 pages. [cited by applicant]
Kingma et al., “Adam: A Method for Stochastic Optimization”; https://doi.org/10.48550/arXiv.1412.6980, Dec. 22, 2014, 15 pages. [cited by applicant]
Lampe et al., “Neural architectures for named entity recognition”; https//:doi.org/10.48550/arXiv.1603.01360, Mar. 4, 2016, 11 pages. [cited by applicant]
Liu et al., “ROBERTa: A Robustly Optimized BERT Pretraining Approach”, https/:doi.org/10.48550/arXiv.1907.11692, Jul. 26, 2019, 13 pages. [cited by applicant]
Ratinov et al. “Design challenges and misconceptions in named entity recognition”, Proceedings of the thirteenth conference on computational natural language learning (CoNLL-2009), 2009, pp. 147-155. [cited by applicant]
Smith et al., “Don't Decay the Learning Rate, Increase the Batch Size”, https//:doi.org/10.48550/arXiv.1711.00489; Nov. 1, 2017, 11 pages. [cited by applicant]
Vaswani et al., “Attention Is All You Need”; 2017, https://doi.org/10.48550/arXiv.1706.03762, 15 pages. [cited by applicant]
Wang et al., “Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning”, Jun. 2, 2021, University of Chinese Academy of Sciences, DAMO Academy, Alibaba Group, https://doi.org/10.48550/a… [cited by applicant]