Enhanced conceptual search based on enriched categorizations of item listings
AI-driven techniques are provided for generating multi-modal vector representations that incorporate enriched, model-generated category embeddings to enable conceptual search of item listings. For example, the enhanced conceptualized search can include generating a search vector, using one or more generative artificial intelligence (AI) models, based on a received search query. Some embodiments of generating the search vector may include applying one or more generative AI models to one or more sets of prior user interactions. Based on the search vector, a particular item listing vector can be determined. The item listing vector can be generated by applying one or more generative AI models to each of an image of the item and a textual description of an item listing, and the item listing vector can be generated based on enhanced categories determined (e.g., using one or more generative AI models) for the item listing.
1 . A computer-implemented method comprising:
receiving a search query from a user at a search engine;
determining a plurality of similar users identified as having a particular similarity to the user;
generating a search vector based on the search query and a set of similar user interaction data corresponding to the plurality of similar users identified as having a particular similarity to the user;
determining an item listing vector based on the search vector, the item listing vector of an item listing for an item of a corpus of item listings, the item listing vector generated using one or more generative artificial intelligence models trained to generate the item listing vector based on an image of the item and a textual description of the item, the item listing vector representing at least two categories determined for the item listing, wherein a first category is determined from the textual description, and wherein a second category more narrow than the first category is determined from the image; and
providing a search result for the search query based on determining the item listing vector, wherein the search result is associated with the second category.
2 . The computer-implemented method of claim 1 , wherein the at least two categories are determined by the one or more generative artificial intelligence models from an image aspect from the image of the item and a keyword from the textual description of the item.
3 . The computer-implemented method of claim 1 , the search vector being generated based on a set of prior user interactions of a user associated with the search query, the set of prior user interactions being identified by applying a generative artificial intelligence model to a plurality of prior user interactions of the user.
4 . The computer-implemented method of claim 1 , wherein a plurality of item listing vectors, including the item listing vector of the item listing for the item, are generated by:
determining, by the one or more generative artificial intelligence models, a first category, of the at least two categories, for a feature depicted on a first set of items within item images within the item listings of the corpus;
determining, by the one or more generative artificial intelligence models, a second category, of the at least two categories, for a second feature depicted on the first set of items; and
generating the plurality of item listing vectors for the first set of items based on the first category and the second category.
5 . The computer-implemented method of claim 4 , wherein the second category is based on matching the second feature depicted on the first set of items with a keyword within textual descriptions of the first set of items.
6 . The computer-implemented method of claim 5 , wherein the second category is a broader category than the first category.
7 . The computer-implemented method of claim 4 , the item listing vector further determined based on a set of prior user interactions with one or more of the item listings of the corpus, the set of prior user interactions being identified by applying a generative artificial intelligence model to a plurality of prior user interactions with the one or more of the item listings having the at least two categories including the first category and the second category.
8 . The computer-implemented method of claim 1 , further comprising:
providing a plurality of search results, including the search result, for the search query based on determining the item listing vector;
receiving feedback based on providing the plurality of search results;
modifying the search vector based on the feedback; and
providing a second plurality of search results for the modified search vector.
9 . A computer system comprising:
a processor; and
a computer storage medium storing computer-useable instructions that, when used by the processor, causes the computer system to perform operations comprising:
receiving a search query from a user at a search engine;
generating a search vector based on the search query and a set of similar user interaction data corresponding to a plurality of similar users identified as having a particular similarity to the user;
determining an item listing vector based on the search vector, the item listing vector from a plurality of item listing vectors for a plurality of item listings, each of the plurality of item listing vectors generated using one or more generative artificial intelligence models trained to generate the plurality of item listing vectors based on both an image and a textual description of an associated item, wherein a first category is determined from the textual description, and wherein a second category more narrow than the first category is determined from the image; and
providing a search result for the search query based on determining the item listing vector, wherein the search result is associated with the second category.
10 . The computer system of claim 9 , wherein each of the plurality of item listing vectors are generated by determining, by the one or more generative artificial intelligence models, a first category for a feature depicted on the associated item within the image, the first category for the feature being included within the textual description of the associated item.
11 . The computer system of claim 10 , wherein each of the plurality of item listing vectors are generated by determining, by the one or more generative artificial intelligence models, a second category for another feature depicted on the associated item within the image, the second category for the feature being included within the textual description of the associated item.
12 . The computer system of claim 11 , the search vector being generated based on a set of prior user interactions of a user associated with the search query, the set of prior user interactions being identified by applying a generative artificial intelligence model to a plurality of prior user interactions of the user to identify the set of prior user interactions corresponding to the first category and the second category.
13 . The computer system of claim 9 , the search vector being generated from a textual embedding and an image embedding by applying the one or more generative artificial intelligence models to prior user interactions.
14 . The computer system of claim 9 , wherein each of the plurality of item listing vectors are generated by determining, by the one or more generative artificial intelligence models, a plurality of categories for a feature depicted on the associated item within the image, wherein each of the plurality of categories are a different category than a predominating category for the associated item itself.
15 . One or more non-transitory computer storage media storing computer-useable instructions that, when used by a user device, cause the user device to perform operations, the operations comprising:
determining a plurality of similar users identified as having a particular similarity to a user;
generating a search vector for a search query received from the user based on the search query and a set of similar user interaction data corresponding to the plurality of similar users identified as having a particular similarity to the user; and
executing an item search by identifying, based on the search vector, an item listing vector of an item, the item listing vector being generated from at least one category, the at least one category generated by one or more generative artificial intelligence models using at least one image and a textual description associated with a corresponding item listing, the one or more generative artificial intelligence models trained to determine categories for item features depicted within images of items and textual descriptions of the items, wherein a first category is determined from the textual descriptions, and wherein a second category more narrow than the first category is determined from the at least one image.
16 . The one or more non-transitory computer storage media of claim 15 , wherein the one or more generative artificial intelligence models includes Bidirectional Encoder Representations from Transformers (BERT) and an imaging model.
17 . The one or more non-transitory computer storage media of claim 15 , wherein each of the item listing vectors are generated from a first category and a second category determined from the at least one image and textual description associated with the corresponding item listing, the first category being a broader category than the second category, and wherein the first category and the second category are a different category than a predominating category for the associated item itself.
18 . The one or more non-transitory computer storage media of claim 17 , further comprising regenerating an item listing vector for a first item listing of the plurality of item listings based on applying the one or more generative artificial intelligence models to user interactions with the first item listing.
19 . The one or more non-transitory computer storage media of claim 18 , wherein the user interactions include audio data corresponding to the category.
20 . The one or more non-transitory computer storage media of claim 19 , wherein the first category is a meta-category and the second category is a leaf category.