Generating full designs from text
Systems and methods for generating full designs from text include retrieving a plurality of document templates based on a design prompt, and filtering the document templates based on an image prompt. A document template is then selected based on the filtering, and a document is generated based on the document template and the image prompt. Embodiments are further configured to generate content from one or more of the prompts, where the content is included in the final design document.
1 . A method comprising:
retrieving a plurality of document templates based on a design prompt;
encoding an image prompt to obtain an image prompt embedding;
filtering the plurality of document templates by comparing the image prompt embedding to a template embedding for each of the plurality of document templates, respectively;
selecting a document template based on the filtering; and
generating a document based on the document template and the image prompt.
2 . The method of claim 1 , wherein the retrieving of the plurality of document templates further comprises:
encoding the design prompt to obtain a design prompt embedding; and
comparing the design prompt embedding to a template embedding for each of the plurality of document templates, respectively.
3 . The method of claim 1 , further comprising:
encoding a description of a template to obtain a description embedding;
encoding a caption of a visual element of the template to obtain a caption embedding; and
combining the description embedding and the caption embedding to obtain the template embedding.
4 . The method of claim 1 , further comprising:
receiving a text input for the document, wherein the document is based on the text input.
5 . The method of claim 4 , further comprising:
generating a recommended text field based on the design prompt, wherein the text input corresponds to the recommended text field.
6 . The method of claim 4 , further comprising:
sorting the plurality of document templates based on the text input.
7 . The method of claim 4 , further comprising:
computing a location for the text input based on the text input and the document template, wherein the document includes the text input at the location.
8 . The method of claim 1 , further comprising:
displaying the plurality of document templates; and
receiving a selection input, wherein the document template is selected based on the selection input.
9 . The method of claim 1 , wherein:
the image prompt comprises an image, and wherein the document comprises the image.
10 . The method of claim 1 , wherein:
the image prompt comprises a description of an image.
11 . The method of claim 10 , further comprising:
generating an image based on the image prompt, wherein the document includes the generated image.
12 . A non-transitory computer readable medium storing code, the code comprising instructions executable by a processor to:
retrieve a plurality of document templates based on a design prompt;
encode an image prompt to obtain an image prompt embedding;
filter the plurality of document templates by comparing the image prompt embedding to a template embedding for each of the plurality of document templates, respectively;
select a document template based on the filtering; and
generate a document based on the document template and the image prompt.
13 . The non-transitory computer readable medium of claim 12 , the code further comprising instructions executable by a processor to:
encode the design prompt to obtain a design prompt embedding; and
compare the design prompt embedding to a template embedding for each of the plurality of document templates, respectively, wherein the retrieval of the plurality of document templates is based on the comparison.
14 . The non-transitory computer readable medium of claim 12 , the code further comprising instructions executable by a processor to:
receive a text input for the document, wherein the document is based on the text input.
15 . The non-transitory computer readable medium of claim 14 , the code further comprising instructions executable by a processor to:
generate a recommended text field based on the design prompt, wherein the text input corresponds to the recommended text field.
16 . An apparatus comprising:
at least one processor;
at least one memory including instructions executable by the at least one processor;
a template database storing a plurality of document templates;
an intent component configured to encode a design prompt to obtain a design prompt embedding, and further configured to encode an image prompt to obtain an image prompt embedding;
a filtering component configured to filter the plurality of document templates by comparing the image prompt embedding to a template embedding for each of the plurality of document templates, respectively; and
a document generation component configured to generate a document based on an image prompt and a selected document template of the plurality of document templates.
17 . The apparatus of claim 16 , further comprising:
text field recommender configured to generate a recommended text field based on the design prompt.
18 . The apparatus of claim 16 , further comprising:
an image generation model configured to generate an image based on the image prompt.