IP Library Granted Patent US 12,299,385
Granted Patent B2
US 12,299,385 · App. 18/220,790 · Granted May 13, 2025

Machine content generation

Inventor: Bao Tran (Saratoga, CA)
G06F40/169G06F40/137G06N20/00G06F21/105G06F40/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,385
App. No.
18/220,790
Granted
May 13, 2025
Kind
B2
Abstract

Computerized systems and methods are disclosed to generate a document from one or more first and second text prompts, generating one or more context-sensitive text suggestions using a transformer with an encoder on the text prompts and a decoder that produces a text expansion to provide the context-sensitive text suggestions based on the one or more first and second text prompts by applying generative artificial intelligence with token biased weights for zero-shot, one-shot or some-shot generation of the artificial intelligence context-sensitive text suggestions from the one or more first and second text prompts.

Claims (38)

1. A method to generate a document having a predetermined outline, comprising:

providing a user interface to receive a first text to plan a document structure from the first text, wherein the document structure comprises the predetermined outline;

searching one or more sources to locate one or more second texts matching the first text;

providing the first text and the one or more second texts to automatically expand into one or more artificial intelligence context-sensitive texts using a transformer with a decoder that produces a text expansion to provide the context-sensitive text based on the first text and the one or more second texts by applying generative artificial intelligence with normalization and tokenization with zero-shot, one-shot or some-shot generation of the one or more artificial intelligence context-sensitive texts from the first text and the one or more second texts, wherein the transformer receives a stream of tokens with an attention mask at one or more self-attention layers, and a causal mask is used for the text tokens;

generating the document with the predetermined outline by combining the generated artificial intelligence context-sensitive text based on the predetermined outline and the first text and the one or more second texts, wherein the first text or the one or more second texts comprises a chapter or section overview; and

wherein the generated document comprises the chapter or section outline expanded with the generated artificial intelligence context-sensitive text.

2. The method of claim 1 , wherein the generated document comprises a fiction work, a non-fiction work, a computer readable code, a machine specification, a patent application, a blog, a legal document, a response to a query, a response to a search, a marketing content, or a mechanical description.

3. The method of claim 1 , wherein the document structure comprises one or more figures, wherein each of the one or more figures comprises a brief description of the drawing, a figure description overview, and an artificial-intelligence-generated detailed description of the figure from component texts corresponding to items in the figure.

4. The method of claim 1 , further comprising combining a title and a machine generated background text with the first text and providing the combined title, background, and the first text to a learning machine to synthesize artificial intelligence context-sensitive text.

5. The method of claim 1 , further comprising: extracting one or more reference signs from a figure and annotating each the one or more reference signs with a part name; and

forming an artificial intelligence-generated reference text suggestion from the one or more reference signs.

6. The method of claim 1 , wherein the transformer comprises a generative pre-trained transformer (GPT) model or a BERT (Bidirectional Encoder Representations from Transformers) model.

7. The method of claim 1 , further comprising:

determining when component, module, code, data structure, or image perform a similar task and showing the determined component, module, code, data structure, or image to a user and breaking-down the one or more second texts into one or more alternate components with different component text and show the one or more alternate components as an artificial-intelligence-generated design satisfying the first text or the one or more second texts.

8. The method of claim 1 , further comprising:

generating a part list by detecting noun phrases (NPs) in the generated document and corresponding numbers for the NPs.

9. The method of claim 1 , wherein the generated document comprises a response to a query.

10. The method of claim 1 , wherein the generated document comprises a patent application.

11. The method of claim 10 , further comprising from one or more sets of input text, generating artificial intelligence context-sensitive text for at least one of: a title, a background, a summary, a description, additional claims and an abstract.

12. The method of claim 10 , further comprising:

detecting at least one of: an error, an antecedent basis error, a claim support error, and a functional language.

13. The method of claim 10 , further comprising:

generating one or more figures or drawings from a user specification.

14. The method of claim 1 , further comprising:

responding to the first text by searching for one or more responsive documents to supplement the first text and applying the generative artificial intelligence to provide a search result.

15. The method of claim 1 , further comprising: generating a multimedia file, a blog, a book, a story, a legal document, a business document, a plan, or a chatbot response.

16. The method of claim 1 , further comprising:

receiving a document as a the first text and applying generative artificial intelligence to generate a response to a query based on the document.

17. The method of claim 1 , further comprising iteratively searching the first text or the one or more second texts.

18. A system to generate a document having a predetermined outline, the system comprising:

a processor; and

a data storage storing computer readable code, that when executed by the processor, cause the system to perform operations, the operations comprising:

providing a user interface to receive a first text to plan a document structure from the first text, wherein the document structure comprises the predetermined outline;

searching one or more sources to locate one or more second texts matching the first text;

providing the first text and the one or more second texts to automatically expand into one or more artificial intelligence context-sensitive texts using a transformer with a decoder that produces a text expansion to provide the context-sensitive text based on the first text and the one or more second texts by applying generative artificial intelligence with normalization and tokenization with zero-shot, one-shot or some-shot generation of the one or more artificial intelligence context-sensitive texts from the first text and the one or more second texts, wherein the transformer receives a stream of tokens with an attention mask at one or more self-attention layers, and a causal mask is used for the text tokens;

generating the document with the predetermined outline by combining the generated artificial intelligence context-sensitive text based on the predetermined outline and the first text and the one or more second texts, wherein the first text or the one or more second texts comprises a chapter or section overview; and

wherein the generated document comprises the chapter or section outline expanded with the generated artificial intelligence context-sensitive text.

19. The system of claim 18 , wherein the operations further comprises iteratively searching the first text or the one or more second texts.

Continuity (3)
Continuation 17582852 · Jan 24, 2022
Provisional Application 63140774 · Jan 22, 2021
Related Publication 20230351102A1 · Nov 2, 2023
References Cited (37)
US 7716581B2 · Tran · 2010 [cited by applicant]
US 8295471B2 · Spottiswoode · 2012 [cited by applicant]
US 8386350B2 · Plow · 2013 [cited by applicant]
US 9990351B2 · Tran · 2018 [cited by applicant]
US 10195513B2 · Tran · 2019 [cited by applicant]
US 20080004904A1 · Tran · 2008 [cited by applicant]
US 20100145678A1 · Csomai · 2010 [cited by examiner]
US 20120141959A1 · von Ahn Arellano · 2012 [cited by applicant]
US 20130317994A1 · Tran · 2013 [cited by applicant]
US 20160092405A1 · Lee · 2016 [cited by examiner]
US 20180232361A1 · Schick · 2018 [cited by examiner]
US 20190340217A1 · Tran · 2019 [cited by applicant]
US 20200034432A1 · Jain · 2020 [cited by applicant]
US 20200125574A1 · Ghoshal · 2020 [cited by examiner]
US 20200293605A1 · Nelson · 2020 [cited by examiner]
US 20210081411A1 · Begun · 2021 [cited by examiner]
US 20210192140A1 · Galley · 2021 [cited by examiner]
US 20210248474A1 · Baker · 2021 [cited by examiner]
US 20210374603A1 · Xia · 2021 [cited by examiner]
US 20220067486A1 · Klein · 2022 [cited by examiner]
US 20220075929A1 · Booth · 2022 [cited by examiner]
Priya Shree, “The Journey of Open AI GPT models”, Nov. 9, 2020, Walmart Global Tech Blog, pp. 1-20. (Year: 2020). [cited by examiner]
Vaswani et al, Attention Is All You Need, arXiv:1706.03762v5 [cs.CL] Dec. 6, 2017. [cited by applicant]
Devlin et al, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv:1810.04805v2 [cs.CL] May 24, 2019. [cited by applicant]
Horev, BERT Explained: State of the art language model for NLP, https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270 Nov. 10, 2018. [cited by applicant]
Alammar, The Illustrated Transformer, http://jalammar.github.io/illustrated-transformer/ , Jun. 27, 2018. [cited by applicant]
Kitaev et al, Reformer: the Efficient Transformer, ICLR 2020. [cited by applicant]
Beltagy, Longformer: The Long-Document Transformer, arXiv:2004.05150v1 [cs.CL] Apr. 10, 2020. [cited by applicant]
Clark et al, Electra: Pre-Training Text Encoders as Discriminators Rather Than Generators, 2020 International Conference on Learning Representation. [cited by applicant]
Yao, Reformer, Longformer, and Electra: Key Updates to Transformer Architecture in 2020 https://www.topbots.com/key-updates-to-transformer-architecture-2020/, Jun. 2, 2020. [cited by applicant]
Balog et al, Deepcoder: Learning to Write Program, arXiv:1611.01989v2 [cs.LG] Mar. 8, 2017. [cited by applicant]
Ye et al, MISIM: An End-to-End Neural Code Similarity System, arXiv:2006.05265v2 [cs.LG] Jun. 15, 2020. [cited by applicant]
Radford, Language Models are Unsupervised Multitask Learners, https://github.com/openai/gpt-2 2019. [cited by applicant]
Von Platen, How to generate text: using different decoding methods for language generation with Transformers, blog at https://huggingface.co/blog/how-to-generate, Mar. 18, 2020. [cited by applicant]
Roller et a,, Recipes for building an open-domain chatbot, arXiv:2004.13637v2. [cited by applicant]
Kent, Animating gAnime with StyleGAN: Part 1, Oct. 27, 2019 , https://towardsdatascience.com/animating-ganime-with-stylegan-part-1-4cf764578e, Apr. 30, 2020. [cited by applicant]
Kelvin Guu et al, REALM: Retrieval-Augmented Language Model Pre-Training, arXiv:2002.08909 (Feb. 10, 2020) at https://arxiv.org/abs/2002.08909. [cited by applicant]
Cited By (4)
US 12,688,235 US 12,688,364 US 12,705,423 US 12,724,827