Intelligent model selection system for style-specific digital content generation
Aspects of the present disclosure provide systems, methods, and computer-readable storage media that support intelligent model selection for style-specific digital content generation. For example, a system that provides a digital content generation service may include a trained style detection model may receive reference digital content items from a user and extract a user style embedding that represents a style preference of the user. In some implementations, the reference digital content items may include text documents or images provided or selected by the user. The system may compare the user style embedding to a plurality of model style embeddings that each correspond to a respective generative artificial intelligence (AI) model to generate a ranked list of generative AI models. The system may access one or more highest ranked generative AI models from the ranked list to generate novel digital content based on a prompt from the user.
1 . A method for intelligent model selection for style-specific digital content generation, the method comprising:
training, by one or more processors, a style detection model including a neural network using training data that includes labelled digital content items of different styles and corresponding style embeddings;
receiving, by one or more processors, reference data corresponding to one or more reference digital content items that represent a user style preference;
extracting, by the one or more processors using the trained style detection model, a user style embedding from the reference data;
ranking, by the one or more processors, a plurality of model style embeddings based on similarity with the user style embedding to generate a ranked list of generative artificial intelligence (AI) models,
wherein each model style embedding of the plurality of model style embeddings corresponds to a respective generative AI model of a plurality of generative AI models configured to generate digital content based on respective input prompts;
receiving, by the one or more processors, a user input indicating a prompt for content generation;
providing, by the one or more processors, the prompt as input data to one or more highest ranked generative AI models from the ranked list of generative AI models to generate one or more digital content items; and
outputting, by the one or more processors, the one or more digital content items.
2 . The method of claim 1 , wherein ranking the plurality of model style embeddings comprises:
determining, by the one or more processors, for one or more model style embeddings of the plurality of model style embeddings, a respective cosine similarity value based on a comparison of the model style embedding and the user style embedding; and
ranking, by the one or more processors, each of the one or more model style embeddings based on the respective cosine similarity value.
3 . The method of claim 1 , further comprising:
providing, by the one or more processors, one or more prompts as sample input data to the plurality of generative AI models to generate a plurality of sample digital content items; and
extracting, by the one or more processors, the plurality of model style embeddings from the plurality of sample digital content items.
4 . The method of claim 3 , wherein extracting the plurality of model style embeddings comprises:
extracting, by the one or more processors, a first model style embedding based on features of a first set of sample digital content items of the plurality of sample digital content items, the first set of sample digital content items generated by a first generative AI model based on the one or more prompts; and
extracting, by the one or more processors, a second model style embedding based on features of a second set of sample digital content items of the plurality of sample digital content items, the second set of sample digital content items generated by a second generative AI model based on the one or more prompts.
5 . The method of claim 4 , wherein a similarity between the first model style embedding and the second model style embedding corresponds to a similarity between a first content style associated with the first set of sample digital content items and a second content style associated with the second set of sample digital content items.
6 . The method of claim 1 , wherein the one or more reference digital content items comprise one or more reference text documents, wherein the one or more digital content items comprise one or more text documents, and wherein the plurality of generative AI models are trained to generate novel text documents.
7 . The method of claim 6 , wherein extracting the user style embedding comprises:
providing, by the one or more processors, the reference data as input to the trained style detection model that is configured to generate style embeddings and style feature labels for input text documents, the style embeddings comprising feature vectors extracted from the input text documents and the style feature labels comprising labels of one or more style attributes associated with the input text documents.
8 . The method of claim 7 , wherein the trained style detection model is trained to generate the style feature labels based on style-specific training datasets that correspond to the one or more style attributes, and wherein the one or more style attributes comprise formal, informal, active, passive, polite, impolite, biased, neutral, offensive, non-offensive, single author, multiple authors, positive humor, negative humor, factual, or a combination thereof.
9 . The method of claim 7 , further comprising, for each style attribute of the one or more style attributes:
providing, by the one or more processors, a set of prompts that correspond to the style attribute as sample input data to the plurality of generative AI models to generate a plurality of sample text documents; and
extracting, by the one or more processors, model style embeddings that correspond to the style attribute from the plurality of sample text documents.
10 . The method of claim 1 , wherein the one or more reference digital content items comprise one or more reference images, wherein the one or more digital content items comprise one or more images, and wherein the plurality of generative AI models are trained to generate novel images.
11 . The method of claim 10 , wherein extracting the user style embedding comprises:
providing, by the one or more processors, the reference data as input to the trained style detection model that is configured to generate style embeddings for input images;
providing, by the one or more processors, a set of sample prompts as sample input data to the plurality of generative AI models to generate a plurality of sample images; and
extracting, by the one or more processors, the plurality of model style embeddings from the plurality of sample images.
12 . The method of claim 1 , further comprising:
generating, by the one or more processors, corresponding custom training data sets for different style attributes,
wherein each custom training data set of the corresponding custom training data sets includes a set of digital content items associated with a style attribute label of one of the different style attributes;
converting, by the one or more processors, the corresponding custom training data sets into formats mandated for plurality of generative AI models;
training, by the one or more processors, the plurality of generative AI models on at least one of the custom training data sets based on the mandated formats.
13 . A system for intelligent model selection for style-specific digital content generation, the system comprising:
a memory; and
one or more processors communicatively coupled to the memory, the one or more processors configured to:
train a style detection model including a neural network using training data that includes labelled digital content items of different styles and corresponding style embeddings;
receive reference data corresponding to one or more reference digital content items that represent a user style preference;
extract, using the trained style detection model, a user style embedding from the reference data;
rank a plurality of model style embeddings based on similarity with the user style embedding to generate a ranked list of generative artificial intelligence (AI) models,
wherein each model style embedding of the plurality of model style embeddings corresponds to a respective generative AI model of a plurality of generative AI models configured to generate digital content based on respective input prompts;
receive a user input indicating a prompt for content generation;
provide the prompt as input data to one or more highest ranked generative AI models from the ranked list of generative AI models to generate one or more digital content items; and
output the one or more digital content items.
14 . The system of claim 13 , wherein the one or more processors are further configured to:
provide the reference data as input to the trained style detection model that is configured to generate style embeddings and style feature labels for input text documents, the style embeddings comprising feature vectors extracted from the input text documents and the style feature labels comprising labels of one or more style attributes associated with the input text documents.
15 . The system of claim 14 , wherein the trained style detection model comprises:
a natural language processing (NLP) layer configured to generate a plurality of word embeddings based on the input text documents;
a mean pooling layer configured to generate a style embedding based on the plurality of word embeddings; and
a softmax layer configured to generate one or more style feature labels based on the style embedding.
16 . The system of claim 13 , wherein the one or more processors are further configured to:
provide the reference data as input to the trained style detection model that is configured to generate style embeddings for input images.
17 . The system of claim 16 , wherein the trained style detection model comprises a convolutional neural network (CNN) that includes a plurality of convolutional layers, and wherein a style embedding for an input image is based on a plurality of Gram matrices computed based on outputs of the plurality of convolutional layers.
18 . A non-transitory computer-readable storage device comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations for intelligent model selection for style-specific digital content generation, the operations comprising:
training a style detection model including a neural network using training data that includes labelled digital content items of different styles and corresponding style embeddings;
receiving reference data corresponding to one or more reference digital content items that represent a user style preference;
extracting, using the trained style detection model, a user style embedding from the reference data;
ranking a plurality of model style embeddings based on similarity with the user style embedding to generate a ranked list of generative artificial intelligence (AI) models,
wherein each model style embedding of the plurality of model style embeddings corresponds to a respective generative AI model of a plurality of generative AI models configured to generate digital content based on respective input prompts;
receiving a user input indicating a prompt for content generation;
providing the prompt as input data to one or more highest ranked generative AI models from the ranked list of generative AI models to generate one or more digital content items; and
outputting the one or more digital content items.
19 . The non-transitory computer-readable storage device of claim 18 , wherein the operations further comprise:
receiving a user selection of one of the one or more digital content items; and
updating the ranked list of generative AI models based on the user selection.
20 . The non-transitory computer-readable storage device of claim 18 , wherein the one or more reference digital content items comprise one or more reference images, and wherein the operations further comprise:
initiating display of a plurality of reference images for selection by a user to indicate the user style preference; and
receiving a user selection of the one or more reference images from the plurality of reference images.