System, method, and non-transitory computer-readable media using generative artificial intelligence to optimize product search queries

1 . A computer-implemented method comprising:

obtaining a plurality of product representations for a plurality of products, each product representation of each product of the plurality of products comprising a corresponding product description and a corresponding product image of each product;

classifying, by a multi-modal style classification model, each product of the plurality of products into a corresponding style of a plurality of styles based on the corresponding product description and the corresponding product image of each product, wherein the multi-modal style classification model is a machine learning model trained to output styles using input sentence encodings and image feature vectors;

storing, in a knowledge graph, relationships of each product of the plurality of products to other products in the plurality of products based on the corresponding style of each product and at least a portion of the corresponding product description of each product; and

responsive to input selecting a search option from a user interface (UI), communicating a diffusion-generated image that includes a set of products from the plurality of products to display the diffusion-generated image via the Ul by:

determining the set of products for a particular style of the diffusion-generated image by instructing a language model to output the set of products for the particular style based on the relationships in the knowledge graph;

after determining the set of products for the particular style, computing, by a color consistency scoring model configured to compute vectors from color palettes of images, an Euclidian distance between color palette vectors associated with a first product and a second product of the set of products; and

generating, by prompting a text-to-image diffusion model with each corresponding product image of each product of the set of products and instructions to generate the diffusion-generated image that includes the set of products, the diffusion-generated image for display only when the Euclidian distance between the color palette vectors is within a threshold color consistency score.

2 . The computer-implemented method of claim 1 , further comprising:

obtaining a plurality of subsets of complementary products, each product of the plurality of products within a corresponding subset of complementary products of the plurality of subsets of complementary products based on a corresponding type of each product in the corresponding product description of each product; and

storing, in the knowledge graph, the relationships of each product of the plurality of products to the other products in the plurality of products further based on a first association of the corresponding type of each product with the corresponding subset of complementary products and a second association of the corresponding style of each product with the corresponding subset of complementary products.

3 . The computer-implemented method of claim 1 , further comprising:

extracting, by a color palette extraction engine, a corresponding color palette for each product of the plurality of products based on the corresponding product image of each product;

storing, in the knowledge graph, the corresponding color palette of each product of the plurality of products;

computing, by the color consistency scoring model, a corresponding color consistency score between each pair of products of the plurality of products; and

determining, by the language model, the set of products further based on the corresponding color consistency score between each pair of products of the set of products.

4 . The computer-implemented method of claim 1 , further comprising:

generating, by the language model, a suggested textual prompt based on the relationships of each product of the plurality of products to the other products of the plurality of products, the suggested textual prompt comprising at least one of a corresponding product type, a corresponding color of the corresponding product type, a corresponding subset of complementary products, and the corresponding style; and

generating, by the text-to-image diffusion model, the diffusion-generated image that includes the set of products of the plurality of products further based on the suggested textual prompt.

5 . The computer-implemented method of claim 1 , further comprising:

obtaining customer data of a customer, the customer data comprising at least one of images saved by the customer, purchases by the customer, and search queries of the customer;

determining, by a style affinity scoring model, a customer style affinity score for the customer based on the customer data of the customer; and

determining, by the language model, the set of products further based on the customer style affinity score of the customer.

6 . The computer-implemented method of claim 1 , wherein the multi-modal style classification model is a deep learning model trained based on training products, wherein each of the training products comprise a product description, a product image, and a labeled style of the plurality of styles.

7 . The computer-implemented method of claim 1 , wherein the multi-modal style classification model receives an input vector comprising a product description sentence encoding based on a product description, a product title sentence encoding based on the product description, and a product image feature set generated from a convolutional neural network based on the corresponding product image to classify each product into the corresponding style.

8 . One or more non-transitory computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

obtaining a plurality of product representations for a plurality of products, each product representation of each product of the plurality of products comprising a corresponding product description and a corresponding product image of each product;

classifying, by a multi-modal style classification model, each product of the plurality of products into a corresponding style of a plurality of styles based on the corresponding product description and the corresponding product image of each product, wherein the multi-modal style classification model is a machine learning model trained to output styles using input sentence encodings and image feature vectors;

storing, in a knowledge graph, relationships of each product of the plurality of products to other products in the plurality of products based on the corresponding style of each product and at least a portion of the corresponding product description of each product; and

responsive to input selecting a search option from a user interface (UI), communicating a suggested textual prompt comprising a textual description of a set of products from the plurality of products and a diffusion-generated image that includes the set of products from the plurality of products to display the suggested textual prompt and the diffusion-generated image via the UI by:

determining the set of products for a particular style of the suggested textual prompt and the diffusion-generated image by instructing a language model to output the set of products for the particular style based on the relationships in the knowledge graph;

after determining the set of products for the particular style, computing, by a color consistency scoring model configured to compute vectors from color palettes of images, an Euclidian distance between color palette vectors associated with a first product and a second product of the set of products;

generating, by prompting the language model, the suggested textual prompt for display only when the Euclidian distance between the color palette vectors is within a threshold color consistency score; and

generating, by prompting a text-to-image diffusion model with each corresponding product image of each product of the set of products and instructions to generate the diffusion-generated image that includes the set of products, the diffusion-generated image for display only when the Euclidian distance between the color palette vectors is within the threshold color consistency score.

9 . The one or more non-transitory computer-readable media of claim 8 , wherein the method further comprises:

obtaining a plurality of subsets of complementary products, each product of the plurality of products within a corresponding subset of complementary products of the plurality of subsets of complementary products based on a corresponding type of each product in the corresponding product description of each product; and

storing, in the knowledge graph, the relationships of each product of the plurality of products to the other products in the plurality of products further based on a first association of the corresponding type of each product with the corresponding subset of complementary products and a second association of the corresponding style of each product with the corresponding subset of complementary products.

10 . The one or more non-transitory computer-readable media of claim 8 , wherein the method further comprises:

extracting, by a color palette extraction engine, a corresponding color palette for each product of the plurality of products based on the corresponding product image of each product;

storing, in the knowledge graph, the corresponding color palette of each product of the plurality of products;

computing, by the color consistency scoring model, a corresponding color consistency score between each pair of products of the plurality of products; and

determining, by the language model, the set of products further based on the corresponding color consistency score between each pair of products of the set of products.

11 . The one or more non-transitory computer-readable media of claim 8 , wherein the method further comprises:

generating, by prompting the text-to-image diffusion model with each corresponding product image of each product of the set of products and instructions to generate the diffusion-generated image that includes the set of products further based on the suggested textual prompt.

12 . The one or more non-transitory computer-readable media of claim 8 , wherein the method further comprises:

obtaining customer data of a customer, the customer data comprising at least one of images saved by the customer, purchases by the customer, and search queries of the customer;

determining, by a style affinity scoring model, a customer style affinity score for the customer based on the customer data of the customer; and

determining, by the language model, the set of products further based on the customer style affinity score of the customer.

13 . The one or more non-transitory computer-readable media of claim 8 , wherein the method further comprises:

receiving a search query comprising at least one of an image or text; and

determining, by the language model, the set of products further based on the search query.

14 . A computing system comprising:

a processor; and

a non-transitory computer-readable medium having stored thereon instructions that when executed by the processor, cause the processor to perform operations including:

obtaining a plurality of product representations for a plurality of products, each product representation of each product of the plurality of products comprising a corresponding product description and a corresponding product image of each product;

obtaining a plurality of subsets of complementary products, each product of the plurality of products within a corresponding subset of complementary products of the plurality of subsets of complementary products based on a corresponding product type of each product in the corresponding product description of each product;

classifying, by a multi-modal style classification model, each product of the plurality of products into a corresponding style of a plurality of styles based on the corresponding product description and the corresponding product image of each product, wherein the multi-modal style classification model is a machine learning model trained to output styles using input sentence encodings and image feature vectors;

storing, in a knowledge graph, relationships of each product of the plurality of products to other products in the plurality of products based on a first association of the corresponding product type of each product with the corresponding subset of complementary products and a second association of the corresponding style of each product with the corresponding subset of complementary products;

responsive to input selecting a search option from a user interface (UI), communicating a diffusion-generated image that includes a set of products from the plurality of products to display the diffusion-generated image on the Ul by:

determining the set of products for a particular style of the diffusion-generated image by instructing a language model to output the set of products for the particular style based on the relationships in the knowledge graph;

after determining the set of products for the particular style, computing, by a color consistency scoring model configured to compute vectors from color palettes of images, an Euclidian distance between color palette vectors of at least two products of the set of products; and

generating, by prompting a text-to-image diffusion model with each corresponding product image of each product of the set of products and instructions to generate the diffusion-generated image that includes the set of products, the diffusion-generated image for display only when the Euclidian distance between the color palette vectors is within a threshold color consistency score; and

responsive to an indication of a product in the set of products of the diffusion-generated image, communicating a set of different products for display based on a relationship of the product in the set of products to the set of different products.

15 . The computing system of claim 14 , wherein the indication is at least one of a hover action or a selection action.

16 . The computing system of claim 14 , wherein the set of different products is based on at least one of the corresponding product type or the corresponding subset of complementary products of the product in the set of products of the diffusion-generated image.

17 . The computing system of claim 14 , wherein responsive to the indication of the product in the set of products of the diffusion-generated image communicating the set of different products for display further comprises:

receiving the indication of the product in the set of products of the diffusion-generated image;

generating a set of products attributes of the product;

receiving a selection of a change to a product attribute of the set of products attributes of the product; and

generating the set of different products further based on the change to the product attribute.

18 . The computing system of claim 14 , the operations further including:

extracting, by a color palette extraction engine, a corresponding color palette for each product of the plurality of products based on the corresponding product image of each product;

storing, in the knowledge graph, the corresponding color palette of each product of the plurality of products;

computing, by the color consistency scoring model, a corresponding color consistency score between each pair of products of the plurality of products;

determining, by the language model, the set of products further based on the corresponding color consistency score between each pair of products of the set of products; and

responsive to receiving a further indication for a different color of the product, communicating the set of different products for display based on the different color.

19 . The computing system of claim 14 , the operations further including:

generating, by the language model, a suggested textual prompt based on the relationships of each product of the plurality of products to the other products of the plurality of products, the suggested textual prompt comprising at least one of the corresponding product type, a corresponding color of the corresponding product type, the corresponding subset of complementary products, and the corresponding style; and

generating, by the text-to-image diffusion model, the diffusion-generated image that includes the set of products of the plurality of products further based on the suggested textual prompt.

20 . The computing system of claim 14 , the operations further including:

obtaining customer data of a customer, the customer data comprising at least one of images saved by the customer, purchases by the customer, and search queries of the customer;

determining, by a style affinity scoring model, a customer style affinity score for the customer based on the customer data of the customer; and

determining, by the language model, the set of products further based on the customer style affinity score of the customer.