Automatic generation of presentations from documents
Embodiments of the present disclosure include extracting structured text from a source document. The structured text comprises a plurality of source sections. Some embodiments generate a semantic outline based on the structured text. In some examples, the semantic outline comprises a plurality of output headings. Some embodiments generate text content corresponding to each of the plurality of output headings. An image is selected from the source document for each of the plurality of output headings by computing a similarity score between the image and the text content. Then, an output document is generated based on the semantic outline, where the output document comprises a plurality of output sections corresponding to the plurality of output headings, respectively.
1 . A method comprising:
extracting structured text from a source document, wherein the structured text comprises a plurality of source sections;
generating, using a language generation model comprising a transformer model, a semantic outline based on the structured text by performing a self-attention mechanism that combines information from the structured text with an input prompt, wherein the input prompt comprises instructions to a plurality of output headings for the semantic outline from the plurality of source sections, and wherein the self-attention mechanism is performed by an encoder or a decoder of the transformer model;
generating, using the language generation model, text content corresponding to each of the plurality of output headings;
selecting an image from the source document for each of the plurality of output headings by computing a similarity score between the image and the generated text content of a respective output heading of the plurality of output headings; and
generating an output document based on the semantic outline, wherein the output document comprises the selected image, the text content, and a plurality of output sections corresponding to the plurality of output headings, respectively.
2 . The method of claim 1 , wherein
the input prompt includes the instructions to generate the semantic outline to cover important sections of the plurality of source sections.
3 . The method of claim 1 , further comprising:
extracting the image from the source document; and
selecting the image to represent a heading of the plurality of output headings, wherein the output document includes the selected image in an output section corresponding to the heading.
4 . The method of claim 1 , further comprising:
identifying a plurality of images from the source document and a pre-determined selection factor; and
filtering the plurality of images based on the pre-determined selection factor to obtain a filtered set of images, wherein the filtered set of images includes the selected image.
5 . The method of claim 1 , wherein selecting the image comprises:
generating a multi-modal text embedding based on the text content;
generating a multi-modal image embedding based on the image; and
computing the similarity score by comparing the multi-modal text embedding and the multi-modal image embedding.
6 . The method of claim 1 , further comprising:
generating a synthesized image based on a heading of the plurality of output headings, wherein the output document includes the synthesized image in an output section corresponding to the heading.
7 . The method of claim 1 , wherein:
the semantic outline includes a plurality of sub-headings for a heading of the plurality of output headings.
8 . The method of claim 1 , wherein:
a number of the plurality of source sections is greater than a number of the plurality of output headings.
9 . The method of claim 1 , further comprising:
identifying a predetermined number of output headings in the output document, wherein the semantic outline is generated based on the predetermined number of output headings.
10 . The method of claim 1 , wherein generating the output document comprises:
obtaining a document template; and
inserting the plurality of output headings into the document template.
11 . The method of claim 1 , wherein:
the output document comprises a slide presentation including a plurality of slides corresponding to the plurality of output sections, respectively.
12 . The method of claim 1 , further comprising:
receiving user feedback on the output document; and
updating the output document based on the user feedback.
13 . An apparatus comprising:
at least one processor;
at least one memory including instructions executable by the at least one processor;
an extraction component comprising parameters stored in the at least one memory and configured to extract structured text from a source document, wherein the structured text comprises a plurality of source sections;
a language generation model comprising a transformer model and including parameters stored in the at least one memory, wherein the language generation model is configured to generate a semantic outline based on the structured text by performing a self-attention mechanism that combines information from the structured text with an input prompt, wherein the input prompt comprises instructions to select a plurality of output headings for the semantic outline from the plurality of source sections, wherein the self-attention mechanism is performed by an encoder or a decoder of the transformer model, and wherein the language generation model is configured to generate text content corresponding to each of the plurality of output headings;
an image selection component configured to select an image from the source document for each of the plurality of output headings by computing a similarity score between the image and the generated text content of a respective output heading of the plurality of output headings; and
a document generator comprising parameters stored in the at least one memory and configured to generate an output document based on the semantic outline, wherein the output document comprises the text content and a plurality of output sections corresponding to the plurality of output headings, respectively.
14 . The apparatus of claim 13 , wherein:
the image selection component selects the image to represent a heading of the plurality of output headings, wherein the output document includes the selected image in an output section corresponding to the heading.
15 . The apparatus of claim 13 , further comprising:
an image generator configured to generate a synthesized image based on a heading of the plurality of output headings, wherein the output document includes the synthesized image in an output section corresponding to the heading.
16 . The apparatus of claim 13 , further comprising:
a user interface configured to receive user feedback on the output document.
17 . A non-transitory computer readable medium storing code for natural language processing, the code comprising instructions executable by at least one processor to:
extracting structured text from a source document, wherein the structured text comprises a plurality of source sections;
generating, using a language generation model comprising a transformer model, a semantic outline based on the structured text by performing a self-attention mechanism that combines information from the structured text with an input prompt, wherein the input prompt comprises instructions to select a plurality of output headings for the semantic outline from the plurality of source sections, and wherein the self-attention mechanism is performed by an encoder or a decoder of the transformer model;
generating, using the language generation model, text content corresponding to each of the plurality of output headings;
selecting an image from the source document for each of the plurality of output headings by computing a similarity score between the image and the generated text content of a respective output heading of the plurality of output headings; and
generating an output document based on the semantic outline, wherein the output document comprises the selected image, the text content, and a plurality of output sections corresponding to the plurality of output headings, respectively.
18 . The non-transitory computer readable medium of claim 17 , wherein:
the input prompt includes the instructions to generate the semantic outline to cover important sections of the plurality of source sections.
19 . The non-transitory computer readable medium of claim 17 , the code further comprising instructions executable by the at least one processor to:
identify a predetermined number of output headings in the output document, wherein the semantic outline is generated based on the predetermined number of output headings.