System and method for determining substitutes for a requested product and the order to provide the substitutes

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:

dynamically training a machine learning module to build a product graph relating to a domain;

receiving, via computer network, an inquiry from a user device for a user, wherein the inquiry comprises an image and a text describing a requested product in the domain;

extracting one or more visual attributes of the requested product based on the extracting one or more textual attributes of the requested product based on the text;

determining a first projected requested product, in the product graph, based on the one or more visual attributes of the requested product;

determining a second projected requested product in the product graph based on the one or more textual attributes of the requested product;

determining one or more substitutes for the requested product by:

determining one or more first candidates, each first candidate of the one or more first candidates existing in both the product graph and an inventory database and comprising a visual distance between the first projected requested product and the each first candidate in the product graph;

determining one or more second candidates, each second candidate of the one or more second candidates existing in both the product graph and the inventory database and comprising a textual distance between the second projected requested product and the each second candidate the product graph, the one or more substitutes comprising the one or more first candidates and the one or more second candidates;

determining a score of a candidate substitute of the one or more substitutes based on the visual distance and the textual distance of the candidate substitute of the one or more substitutes, Wherein a greater weight is given to the visual distance than the textual distance in determining the score; and

sorting the one or more substitutes based on the score of the candidate substitute of the one or more substitutes; and

in response to receiving the inquiry from the user, automatically transmitting, via the computer network and to the user device for the user, information about higher ranked ones of the one or more substitutes to facilitate a display of the information on a graphical user interface on the user device,

wherein:

the product graph comprises:

a multi-dimensional space;

one or more product nodes, wherein:

the one or more product nodes comprise the one or more substitutes; and

each of the one or more product nodes comprises a respective product coordinate in the multi-dimensional space;

one or more entity nodes, wherein:

the one or more entity nodes comprise the one or more visual attributes and the one or more textual attributes of the requested product; and

each of the one or more entity nodes comprises a respective entity coordinate in the multi-dimensional space;

one or more entity-entity connections between each pair of related entity nodes of the one or more entity nodes;

one or more product-entity connections, wherein:

each of the one or more product-entity connections is between a product node of the one or more product nodes and an entity node of the one or more entity nodes;

the entity node is related to the product node; and

the each of the one or more product-entity connections comprises a respective product-entity connection distance determined based on a respective relatedness degree between the product node and the entity node; and

one or more product-product connections between a first product node of the one or more product nodes and a second product node of the one or more product nodes, wherein the first product node and the second product node are determined to be substitutable by or complimentary of each other;

determining the one or more first candidates further comprises determining the visual distance between the first projected requested product and each first candidate of the one or more first candidates in the product graph based on a respective first vector between the first projected requested product and the each first candidate as projected in each dimension of multiple dimensions of the multi-dimensional space; and

determining the one or more second candidates further comprises determining the textual distance between the second projected requested product and each second candidate of the one or more second candidates in the product graph based on a respective second vector between the second projected requested product and the each second candidate as projected in each dimension of the multiple dimensions of the multi-dimensional space.

2. The system in claim 1 , wherein the computing instructions are further configured to run on the one or more processors and perform:

extracting one or more color attributes of the requested product based on the image and the text; and

determining a color distance of the candidate substitute of the one or more substitutes based on the one or more color attributes of the requested product and one or more color entities of the candidate substitute of the one or more substitutes;

wherein:

the one or more entity nodes of the product graph further comprise the one or more color attributes of the requested product and the one or more color entities of the candidate substitute of the one or more substitutes; and

the determining the score of the candidate substitute of the one or more substitutes further comprises determining the score of the candidate substitute of the one or more substitutes based on the color distance of the candidate substitute of the one or more substitutes.

3. The system in claim 2 , wherein:

the one or more color attributes of the requested product comprises a product major color and a product minor color; and

the one or more color entities of the candidate substitute of the one or more substitutes comprise a candidate major color and a candidate minor color.

4. The system in claim 3 , wherein:

the score of the candidate substitute of the one or more substitutes is determined by:

α

I

⁢

1

d

+

β

C

⁡

(

1

d

m

⁢

a

⁢

j

-

c

+

1

d

m

⁢

t

⁢

n

-

c

)

+

γ

T

⁢

1

d

T

;

wherein:

d 1 : the visual distance between the first projected requested product and the candidate substitute of the one or more substitutes;

d t : the textual distance between the second projected requested product and the candidate substitute of the one or more substitutes;

f maj-c : a major color distance between the product major color and the candidate major color;

d min-c : a minor color distance between the product minor color and the candidate minor color;

α 1 : a predetermined image weight;

β C : a predetermined color weight; and

Y T : a predetermined text weight.

5. The system in claim 2 , wherein the computing instructions are further configured to run on the one or more processors and perform:

determining a context of the inquiry;

wherein:

the context of the inquiry includes one of an address of the user, a purchase history of the user, another term in the inquiry of the user, a style identified in the image, or another object identified in the image;

the extracting the one or more visual attributes of the requested product further comprises extracting the one or more visual attributes based on the context;

the extracting the one or more textual attributes of the requested product further comprises extracting the one or more textual attributes based on the context; and

the extracting the one or more color attributes of the requested product further comprises extracting the one or more color attributes based on the context.

6. The system in claim 1 , wherein:

the one or more first candidates comprise a predetermined number of product nodes of the one or more product nodes that comprise lowest visual distances from the first projected requested product among visual distances of the one or more product nodes; and

the one or more second candidates comprise the predetermined number of product nodes of the one or more product nodes that comprise lowest textual distances from the second projected requested product among textual distances of the one or more product nodes.

7. The system in claim 1 , wherein the machine learning module is configured to:

dynamically learn the information about the domain from one or more expert opinions, the inventory database, and public records;

create one or more product categories in the domain based on the information about the domain;

extract one or more first entity nodes from the one or more product categories based on the information about the domain;

create one or more second entity nodes based on the one or more first entity nodes, each of the one or more second entity nodes being related to at least one of the one or more first entity nodes according to a first set of criteria, wherein:

the one or more entity nodes comprise the one or more first entity nodes and the one or more second entity nodes;

build an entity-entity connection, of the one or more entity-entity connections, between two related entity nodes of the one or more entity nodes, when the two related entity nodes are either associated with one of the one or more product categories or related to each other according to the first set of criteria;

build a product-entity connection with the respective product-entity connection distance, of the one or more product-entity connections, between a first entity node of the one or more entity nodes and a first product node of the one or more product nodes, based on the information about the domain, wherein:

the first product node is related to the first entity node; and

the respective product-entity connection distance of the product-entity connection is determined based on the respective relatedness degree between the first entity node and the first product node; and

build a product-product connection of the one or more product-product connections, between a first product node and a second product node selected from the one or more product nodes when the first product node and the second product node are determined to be substitutable by or complimentary of each other based on the information about the domain.

8. The system in claim 7 , wherein the computing instructions are further configured to run on the one or more processors and perform:

providing a user interface configured to be executed on one or more user computers for one or more advanced users to receive the one or more expert opinions from the one or more advanced users;

wherein:

the public records include at least one of product catalogs, publications, product reviews, blog entries, online discussion threads, search keywords, curated collections, or purchase patterns.

9. The system in claim 7 , wherein the first set of criteria used by the machine learning module to determine that a first entity node of the one or more entity nodes is related to a second entity node of the one or more entity nodes comprises:

the first entity node is linguistically similar to the second entity node;

the first entity node is linguistically opposite to the second entity node; and

the first entity node is linguistically inferable from the second entity node.

10. The system in claim 7 , wherein the machine learning module is further configured to assign a distance value of each of the entity-entity connection and the product-product connection.

11. A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:

dynamically training a machine learning module to build a product graph relating to a domain;

receiving, via a computer network, an inquiry from a user device for a user, wherein the inquiry comprises an image and a text describing a requested product in the domain;

extracting one or more visual attributes of the requested product based on the image;

extracting one or more textual attributes of the requested product based on the text;

determining a first projected requested product, in the product graph, based on the one or more visual attributes of the requested product;

determining a second projected requested product in the product graph based on the one or more textual attributes of the requested product;

determining one or more substitutes for the requested product by:

determining one or more first candidates, each first candidate of the one or more first candidates existing in both the product graph and an inventory database and comprising a visual distance between the first projected requested product and the each first candidate in the product graph;

determining one or more second candidates, each second candidate of the one or more second candidates existing in both the product graph and the inventory database and comprising a textual distance between the second projected requested product and the each second candidate in the product graph, the one or more substitutes comprising the one or more first candidates and the one or more second candidates;

determining a score of a candidate substitute of the one or more substitutes based on the visual distance and the textual distance of the candidate substitute of the one or more substitutes, wherein a greater weight is given to the visual distance than the textual distance in determining the score; and

sorting the one or more substitutes based on the score of the candidate substitute of the one or more substitutes; and

in response to receiving the inquiry from the user, automatically transmitting, via the computer network and to the user device for the user, information about higher ranked ones of the one or more substitutes to facilitate a display of the information on a graphical user interface on the user device,

wherein:

the product graph comprises:

a multi-dimensional space;

one or more product nodes, wherein:

the one or more product nodes comprise the one or more substitutes; and

each of die one or more product nodes comprises a respective product coordinate in the multi-dimensional space;

one or more entity nodes, wherein:

the one or more entity nodes comprise the one or more visual attributes and the one or more textual attributes of the requested product; and

each of the one or more entity nodes comprises a respective entity coordinate in the multi-dimensional space;

one or more entity-entity connections between each pair of related entity nodes of the one or more entity nodes;

one or more product-entity connections, wherein:

each of the one or more product-entity connections is between a product node of the one or more product nodes and an entity node of the one or more entity nodes:

the entity node is related to the product node; and

the each of the one or more product-entity connections comprises a respective product-entity connection distance determined based on a respective relatedness degree between the product node and the entity node; and

one or more product-product connections between a first product node of the one or more product nodes and a second product node of the one or more product nodes, wherein the first product node and the second product node are determined to be substitutable by or complimentary of each other;

determining the one or more first candidates further comprises determining the visual distance between the first projected requested product and each first candidate of the one or more first candidates in the product graph based on a respective first vector between the first projected requested product and the each first candidate as projected in each dimension of multiple dimensions of the multi-dimensional space; and

determining the one or more second candidates further comprises determining the textual distance between the second projected requested product and each second candidate of the one or more second candidates in the product graph based on a respective second vector between the second projected requested product and the each second candidate as projected in each dimension of the multiple dimensions of the multi-dimensional space.

12. The method in claim 11 further comprising:

extracting one or more color attributes of the requested product based on the image and the text; and

determining a color distance of the candidate substitute of the one or more substitutes based on the one or more color attributes of the requested product and one or more color entities of the candidate substitute of the one or more substitutes;

wherein:

the one or more entity nodes of the product graph further comprise the one or more color attributes of the requested product and the one or more color entities of the candidate substitute of the one or more substitutes; and

the determining the score of the candidate substitute of the one or more substitutes further comprises determining the score of the candidate substitute of the one or more substitutes based on the color distance of the candidate substitute of the one or more substitutes.

13. The method in claim 12 , wherein:

the one or more color attributes of the requested product comprises a product major color and a product minor color; and

the one or more color entities of the candidate substitute of the one or more substitutes comprise a candidate major color and a candidate minor color.

14. The method in claim 13 , wherein:

the score of the candidate substitute of the one or more substitutes is determined by:

α

I

⁢

1

d

I

+

β

C

⁡

(

1

d

m

⁢

a

⁢

j

-

c

+

1

d

m

⁢

t

⁢

n

-

c

)

+

γ

T

⁢

1

d

T

;

wherein:

d 1 : the visual distance between the first projected requested product and the candidate substitute of the one or more substitutes;

d t : the textual distance between the second projected requested product and the candidate substitute of the one or more substitutes;

d maj-c : a major color distance between the product major color and the candidate major color;

d min-c : a minor color distance between the product minor color and the candidate minor color;

α 1 : a predetermined image weight;

β C : a predetermined color weight; and

Y T : a predetermined text weight.

15. The method in claim 12 further comprising:

determining a context of the inquiry;

wherein:

the context of the inquiry includes one of an address of the user, a purchase history of the user, another term in the inquiry of the user, a style identified in the image, or another object identified in the image;

the extracting the one or more visual attributes of the requested product further comprises extracting the one or more visual attributes based on the context;

the extracting the one or more textual attributes of the requested product further comprises extracting the one or more textual attributes based on the context; and

the extracting the one or more color attributes of the requested product further comprises extracting the one or more color attributes based on the context.

16. The method in claim 11 , wherein:

the one or more first candidates comprise a predetermined number of product nodes of the one or more product nodes that comprise lowest visual distances from the first projected requested product among visual distances of the one or more product nodes; and

the one or more second candidates comprise the predetermined number of product nodes of the one or more product nodes that comprise lowest textual distances from the second projected requested product among textual distances of the one or more product nodes.

17. The method in claim 11 , wherein the machine learning module is configured to:

dynamically learn the information about the domain from one or more expert opinions, the inventory database, and public records;

create one or more product categories in the domain based on the information about the domain;

extract one or more first entity nodes from the one or more product categories based on the information about the domain;

create one or more second entity nodes based on the one or more first entity nodes, each of the one or more second entity nodes being related to at least one of the one or more first entity nodes according to a first set of criteria, wherein:

the one or more entity nodes comprise the one or more first entity nodes and the one or more second entity nodes;

build an entity-entity connection, of the one or more entity-entity connections, between two related entity nodes of the one or more entity nodes, when the two related entity nodes are either associated with one of the one or more product categories or related to each other according to the first set of criteria;

build a product-entity connection with the respective product-entity connection distance, of the one or more product-entity connections, between a first entity node of the one or more entity nodes and a first product node of the one or more product nodes, based on the information about the domain, wherein:

the first product node is related to the first entity node; and

the respective product-entity connection distance of the product-entity connection is determined based on the respective relatedness degree between the first entity node and the first product node; and

build a product-product connection of the one or more product-product connections, between a first product node and a second product node selected from the one or more product nodes when the first product node and the second product node are determined to be substitutable by or complimentary of each other based on the information about the domain.

18. The method in claim 17 further comprising:

providing a user interface configured to be executed on one or more user computers for one or more advanced users to receive the one or more expert opinions from the one or more advanced users;

wherein:

the public records include at least one of product catalogs, publications, product reviews, blog entries, online discussion threads, search keywords, curated collections, or purchase patterns.

19. The method in claim 17 , wherein the first set of criteria used by the machine learning module to determine that a first entity node of the one or more entity nodes is related to a second entity node of the one or more entity nodes comprises:

the first entity node is linguistically similar to the second entity node;

the first entity node is linguistically opposite to the second entity node; and

the first entity node is linguistically inferable from the second entity node.

20. The method in claim 17 , wherein the machine learning module is further configured to assign a distance value of each of the entity-entity connection and the product-product connection.