Systems and methods for product retrieval
Systems and methods for product retrieval are described. One or more aspects of the systems and methods include receiving a query that includes a text description of a product associated with a brand; identifying the product based on the query by comparing the text description to a product embedding of the product, wherein the product embedding is based on a brand embedding of the brand; and displaying product information for the product in response to the query, wherein the product information includes the brand.
1 . A method for product retrieval, comprising:
receiving a query that includes a text description of a target result;
encoding the query to obtain a text embedding representing the target result;
generating a product embedding for a matching product by encoding metadata for the matching product using a plurality of entity encoders to obtain a plurality of entity embeddings and combining the plurality of entity embeddings to obtain the product embedding, wherein each of the plurality of entity embeddings corresponds to an attribute key of the metadata;
identifying the matching product based on the query by comparing the text embedding to the product embedding of the matching product; and
displaying product information for the matching product in response to the query, wherein the product information includes at least one attribute of the metadata corresponding to the attribute key.
2 . The method of claim 1 , wherein identifying the matching product comprises:
identifying a plurality of candidate products, wherein each of the plurality of candidate products is associated with a candidate product embedding; and
comparing the text embedding to the candidate product embedding for each of the plurality of candidate products.
3 . The method of claim 1 , further comprising:
displaying an image of the matching product in response to the query.
4 . The method of claim 1 , wherein:
the metadata comprises structured metadata including a plurality of key-value pairs.
5 . The method of claim 1 , further comprising:
encoding a product description of the matching product to obtain a product description embedding;
combining the plurality of entity embeddings and the product description embedding to obtain the product embedding.
6 . The method of claim 1 , further comprising:
encoding an image of the matching product to obtain an image embedding; and
combining the image embedding with the plurality of entity embeddings to obtain the product embedding.
7 . A method for product retrieval, comprising:
obtaining a training set by:
encoding, using a multi-modal encoder, an image and the unstructured text describing the image to obtain an image embedding and an image description embedding, respectively,
encoding structured metadata corresponding to the image using a plurality of encoders to obtain a plurality of entity embeddings, wherein each of the plurality of entity embeddings corresponds to an attribute key of the structured metadata, and
combining the plurality of entity embeddings and the image description embedding to obtain a combined embedding;
computing a loss based on the combined embedding and the image embedding; and
training the multi-modal encoder using the training set by updating parameters of the multi-modal encoder based on the loss.
8 . The method of claim 7 , wherein:
the combined embedding is based on a weighted combination of the plurality of entity embeddings.
9 . The method of claim 7 , further comprising:
identifying the image and the unstructured text as a positive sample pair;
identifying an additional image and an additional unstructured text as a negative sample pair, wherein the additional image is not associated with the additional unstructured text; and
computing a contrastive learning loss based on the positive sample pair and the negative sample pair, wherein the loss comprises the contrastive learning loss.
10 . A system comprising:
a memory component; and
a processing device coupled to the memory component, the processing device configured to perform operations comprising:
receiving a query that includes a text description of a target result;
encoding the query to obtain a text embedding representing the target result;
generating a product embedding for a matching product by encoding metadata for the matching product using a plurality of entity encoders to obtain a plurality of entity embeddings and combining the plurality of entity embeddings to obtain the product embedding, wherein each of the plurality of entity embeddings corresponds to an attribute key of the metadata;
identifying the matching product based on the query by comparing the text embedding to the product embedding of the matching product; and
displaying product information for the matching product in response to the query, wherein the product information includes at least one attribute of the metadata corresponding to the attribute key.
11 . The system of claim 10 , further comprising:
an image encoder of a multi-modal encoder configured to encode an image of the matching product to obtain an image embedding.
12 . The system of claim 10 , further comprising:
a text encoder of a multi-modal encoder configured to obtain the text embedding.
13 . The system of claim 10 , wherein:
each of the plurality of entity encoders corresponds to an attribute key of the metadata.
14 . The system of claim 10 , further comprising:
an entity selector configured to weight the plurality of entity embeddings to obtain a weighted entity embedding, wherein the product embedding is based on the weighted entity embedding.