IP Library › Granted Patent US 11,587,139
Granted Patent B2
US 11,587,139 · App. 16/779,545 · Granted Feb 21, 2023

Gender attribute assignment using a multimodal neural graph

Inventors: Mansi Ranjit Mane (Sunnyvale, CA); Anirudha Sundaresan (Sunnyvale, CA); Aditya Mantha (Sunnyvale, CA); Stephen Dean Guo (Saratoga, CA); Kannan Achan (Saratoga, CA)
Assignee: WALMART APOLLO, LLC
G06Q30/0631G06F16/906G06F16/9024G06F40/20G06N3/0481G06Q30/0603
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,587,139
App. No.
16/779,545
Filed
Jan 31, 2020
Granted
Feb 21, 2023
Kind
B2
Art Unit
2161
USPC
706/12
Abstract

A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform receiving from an item catalog database a respective item description and respective attribute values for each item of a set of items; generating text embeddings using a text embedding model to represent the respective item description and the respective attribute values; generating a graph of the set of items from the item catalog database connected by a set of edges; training the text embedding model and a machine learning model using a neural loss function based on the graph; and automatically determining, based on the machine learning model, as trained, a gender label for each first item in which the gender classification is unlabeled and in which a respective quantity of respective attribute values for the each first item is at least a predetermined threshold. Other embodiments are disclosed.

Claims (56)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:

receiving from an item catalog database a respective item description and respective attribute values for each item of a set of items, wherein a gender classification of the respective attribute values for the each item of the set of items is either labeled or unlabeled;

generating text embeddings using a text embedding model to represent the respective item description and the respective attribute values for the each item of the set of items;

generating a graph of the set of items from the item catalog database connected by a set of edges, wherein each pair of items of the set of items that is connected by a respective edge of the set of edges in the graph has been viewed together in one or more respective sessions, the respective edge comprises a weight comprising a co-view count, and the set of edges comprises (a) one or more unlabeled-unlabeled edges, (b) one or more labeled-unlabeled edges, and (c) one or more labeled-labeled edges;

training the text embedding model and a machine learning model using a neural loss function based on the graph; and

automatically determining, based on the machine learning model, as trained, a gender label for each first item of the set of items in which the gender classification is unlabeled and in which a respective quantity of respective attribute values for the each first item is at least a predetermined threshold.

2. The system of claim 1 , wherein the computing instructions are further configured to perform:

determining, based on an image embedding model, as trained, a gender label for each second item of the set of items that does not meet the predetermined threshold.

3. The system of claim 1 , wherein the predetermined threshold is 5.

4. The system of claim 1 , wherein the computing instructions are further configured to perform:

transforming an image into a second vector representing the image using a residual neural network (“ResNet”).

5. The system of claim 1 , wherein the computing instructions are further configured to perform:

training an image embedding model based on images of items from the item catalog database using loss equations to minimize a distance between text representations and image representations for the items.

6. The system of claim 5 , wherein the images depict items of clothing from the item catalog database.

7. The system of claim 1 , wherein:

the text embedding model is a Bidirectional Encoder Representations from Transformers (“BERT”); and

an output from the text embedding model comprises a vector representation.

8. The system of claim 1 , wherein training the text embedding model and the machine learning model using the neural loss function based on the graph further comprises:

training the machine learning model with the neural loss function based on first distances between first text embeddings for first pairs of nodes connected by the one or more labeled-labeled edges, second distances between second text embeddings for second pairs of nodes connected by the one or more labeled-unlabeled edges, third distances between third text embeddings for third pairs of nodes connected by the one or more unlabeled-unlabeled edges, and a softmax loss cost function for fourth text embeddings of nodes of the graph that are labeled.

9. The system of claim 1 , wherein the gender classification, when labeled, comprises one of:

a male gender label;

a female gender label; or

a unisex gender label.

10. The system of claim 1 , wherein the computing instructions are further configured to perform:

receiving a selection of an anchor item from a user, the anchor item comprising a first gender label;

determining one or more recommended items that match the first gender label based on the gender labels determined by the machine learning model; and

sending instructions to display the one or more recommended items to the user.

11. A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:

receiving from an item catalog database a respective item description and respective attribute values for each item of a set of items, wherein a gender classification of the respective attribute values for the each item of the set of items is either labeled or unlabeled;

generating text embeddings using a text embedding model to represent the respective item description and the respective attribute values for the each item of the set of items;

generating a graph of the set of items from the item catalog database connected by a set of edges, wherein each pair of items of the set of items that is connected by a respective edge of the set of edges in the graph has been viewed together in one or more respective sessions, the respective edge comprises a weight comprising a co-view count, and the set of edges comprises (a) one or more unlabeled-unlabeled edges, (b) one or more labeled-unlabeled edges, and (c) one or more labeled-labeled edges;

training the text embedding model and a machine learning model using a neural loss function based on the graph; and

automatically determining, based on the machine learning model, as trained, a gender label for each first item of the set of items in which the gender classification is unlabeled and in which a respective quantity of respective attribute values for the each first item is at least a predetermined threshold.

12. The method of claim 11 , further comprising:

determining, based on an image embedding model, as trained, a gender label for each second item of the set of items that does not meet the predetermined threshold.

13. The method of claim 11 , wherein the predetermined threshold is 5.

14. The method of claim 11 , further comprising:

transforming an image into a second vector representing the image using a residual neural network (“ResNet”).

15. The method of claim 11 , further comprising:

training an image embedding model based on images of items from the item catalog database using loss equations to minimize a distance between text representations and image representations for the items.

16. The method of claim 15 , wherein the images depict items of clothing from the item catalog database.

17. The method of claim 11 , wherein:

the text embedding model is a Bidirectional Encoder Representations from Transformers (“BERT”); and

an output from the text embedding model comprises a vector representation.

18. The method of claim 11 , wherein training the text embedding model and the machine learning model using the neural loss function based on the graph further comprises:

training the machine learning model with the neural loss function based on first distances between first text embeddings for first pairs of nodes connected by the one or more labeled-labeled edges, second distances between second text embeddings for second pairs of nodes connected by the one or more labeled-unlabeled edges, third distances between third text embeddings for third pairs of nodes connected by the one or more unlabeled-unlabeled edges, and a softmax loss cost function for fourth text embeddings of nodes of the graph that are labeled.

19. The method of claim 11 , wherein the gender classification, when labeled, comprises one of:

a male gender label;

a female gender label; or

a unisex gender label.

20. The method of claim 11 , further comprising:

receiving a selection of an anchor item from a user, the anchor item comprising a first gender label;

determining one or more recommended items that match the first gender label based on the gender labels determined by the machine learning model; and

sending instructions to display the one or more recommended items to the user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2020
From: MANE, MANSI RANJIT; SUNDARESAN, ANIRUDHA; MANTHA, ADITYA; GUO, STEPHEN DEAN; ACHAN, KANNAN
To: WALMART APOLLO, LLC
Reel/Frame 052211/0762 →
Continuity (1)
Related Publication 20210241350A1 · Aug 5, 2021
Cited By (1)
US 12,699,733