IP Library › Granted Patent US 12,190,416
Granted Patent B1
US 12,190,416 · App. 18/798,014 · Granted Jan 7, 2025

Method and apparatus for contextually intelligent graph generation

Inventors: Andrei Papancea (New York, NY); Vlad Papancea (New York, NY); Peter Szerzo (Berlin, DE)
Assignee: NLX Inc.
G06T11/206G06F40/40G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,190,416
App. No.
18/798,014
Filed
Aug 8, 2024
Granted
Jan 7, 2025
Kind
B1
Examiner
VO, HUYEN X
Art Unit
2656
USPC
704/9
Abstract

In an embodiment, a representation of a first natural language prompt is received. A representation of the first natural language prompt is sent to a large language model (LLM) configured to identify a first pattern based on the first natural language prompt. A first main inlet associated with the first pattern at a graph, a first success outlet associated with the first pattern at the graph, and a first error outlet associated with the first pattern at the graph is determined based on the graph. A first modification is determined based on the first main inlet, the first success outlet, and the first error outlet. The graph is modified based on the first modification to generate a first modified graph.

Claims (62)

1. A method, comprising:

receiving, via a processor, a representation of a first natural language prompt;

sending, via the processor, a representation of the first natural language prompt to a large language model (LLM) configured to identify a first pattern based on the first natural language prompt;

determining, via the processor and based on a graph, a first main inlet associated with the first pattern at the graph, a first success outlet associated with the first pattern at the graph, and a first error outlet associated with the first pattern at the graph;

determining a first modification based on the first main inlet, the first success outlet, and the first error outlet; and

modifying the graph based on the first modification to generate a first modified graph.

2. The method of claim 1 , wherein the first natural language prompt is received from a compute device, the method further comprising:

causing the first modified graph to be displayed via at least one of a no-code user interface at the compute device or a low-code user interface at the compute device.

3. The method of claim 1 , further comprising:

receiving, via the processor, a representation of a second natural language prompt;

sending, via the processor, a representation of the second natural language prompt to the LLM configured to identify a second pattern based on the second natural language prompt;

determining, via the processor, based on the first modified graph and not based on the graph, a second main inlet associated with the second pattern at the first modified graph, a second success outlet associated with the second pattern at the first modified graph, and a second error outlet associated with the second pattern at the first modified graph;

determining, via the processor, a second modification based on the second main inlet, the second success outlet, and the second error outlet; and

modifying, via the processor, the first modified graph and not the graph based on the second modification to generate a second modified graph.

4. The method of claim 1 , wherein the LLM is further configured to identify a second pattern based on the first natural language prompt, the method further comprising:

determining, using the first modified graph, a second main inlet associated with the second pattern at the first modified graph, a second success outlet associated with the second pattern at the first modified graph, and a second error outlet associated with the second pattern at the first modified graph;

determining a second modification based on the second main inlet, the second success outlet, and the second error outlet; and

modifying the first modified graph based on the second modification to generate a second modified graph.

5. The method of claim 1 , wherein the LLM is further configured to identify a second pattern based on the first natural language prompt, the method further comprising:

determining, via the processor and using the graph, a second main inlet associated with the second pattern at the graph, a second success outlet associated with the second pattern at the graph, and a second error outlet associated with the second pattern at the graph; and

determining, via the processor, a second modification based on the second main inlet, the second success outlet, and the second error outlet, the modifying of the graph to generate the first modified graph further based on the second modification.

6. The method of claim 1 , wherein the graph includes a plurality of nodes, the method further comprising:

receiving an indication of a set of nodes from the plurality of nodes selected by a user, the first modification further determined based on the set of nodes.

7. The method of claim 1 , further comprising:

receiving, via the processor, a representation of a second natural language prompt;

sending, via the processor, a representation of the second natural language prompt to the LLM, the LLM configured to generate an output indicating that the second natural language prompt does not include any patterns; and

refraining, via the processor and in response to generating the output indicating that the second natural language prompt does not include any patterns, from modifying at least one of the graph or the first modified graph.

8. A non-transitory processor-readable medium storing code representing instructions to be executed by one or more processors, the instructions comprising code to cause the one or more processors to:

receive serially each natural language prompt from a plurality of natural language prompts;

as each natural language prompt from the plurality of natural language prompts is received and without waiting for remaining natural language prompts from the plurality of natural language prompts to be received, identify at least one pattern based on that natural language prompt by inputting that natural language prompt into a large language model (LLM), a plurality of patterns resulting from the plurality of natural language prompts being input into the LLM;

as each natural language prompt from the plurality of natural language prompts is received and without waiting for remaining natural language prompts from the plurality of natural language prompts to be received, update a graph based on the at least one pattern associated with that natural language prompt, a modified graph resulting from the graph being iteratively updated based on the plurality of patterns; and

reformat the modified graph to generate a reformatted graph, the reformatted graph displayed via a no-code user interface that receives a set of user inputs and generates a set of responses to the set of user inputs generated based on the reformatted graph.

9. The non-transitory processor-readable medium of claim 8 , wherein at least one natural language prompt from the plurality of natural language prompts is received without receiving an indication of at least one node selected by a user and included in the graph.

10. The non-transitory processor-readable medium of claim 8 , wherein the modified graph includes a first number of edge crossings and the reformatted graph includes a second number of edge crossings that is less than the first number of edge crossings.

11. The non-transitory processor-readable medium of claim 8 , wherein the plurality of natural language prompts is received from a remote compute device and the instructions further comprise code to cause the one or more processors to:

cause the reformatted graph to be displayed at the remote compute device.

12. An apparatus comprising:

a memory; and

a processor operatively coupled to the memory, the processor configured to:

receive a first natural language prompt;

input the first natural language prompt into a large language model (LLM) to identify a first pattern;

generate a graph based the first pattern and at a no-code user interface, the graph representing a first conversation flow;

receive a second natural language prompt that includes a representation of a desired modification to the first conversation flow;

input the second natural language prompt into the LLM to identify a second pattern; and

update the graph, at the no-code user interface, based the second pattern and without generating a new graph, to generate a modified graph that represents a second conversation flow that incorporates the desired modification to the first conversation flow.

13. The apparatus of claim 12 , wherein the graph is a first graph and the processor is further configured to:

receive a third natural language prompt;

input the third natural language prompt into the LLM to identify a third pattern; and

generate a second graph that is new and not a modified version of the first graph based the third pattern.

14. The apparatus of claim 13 , wherein the third natural language prompt is received without receiving an indication of any nodes included in the modified graph that have been selected by a user.

15. The apparatus of claim 12 , wherein the processor is further configured to:

receive an indication of at least one node included in the graph and selected by a user, the modified graph further generated based on the at least one node.

16. The apparatus of claim 12 , wherein the processor is further configured to:

determine a main inlet, a success outlet, and an error outlet at the graph, the modified graph further generated based on the main inlet, the success outlet, and the error outlet.

17. The apparatus of claim 12 , wherein the processor is further configured to:

receive a third natural language prompt;

input the third natural language prompt into the LLM to generate an output that indicates the third natural language prompt does not include any patterns; and

in response to generating the output, refrain from updating at least one of the graph or the modified graph.

18. The apparatus of claim 12 , wherein the processor is further configured to:

establish a starting point at the graph, the modified graph further generated based on the starting point.

19. The apparatus of claim 12 , wherein the graph is generated and updated at the no-code user interface without writing code and without using a template.

20. The apparatus of claim 12 , wherein the first pattern is associated with a pattern context definition indicating at least one parameter the first pattern requires.

Assignments (3)
CHANGE OF NAME Recorded May 18, 2026
From: NLX INC.
To: NLX LLC
Reel/Frame 075582/0229 →
BILL OF SALE Recorded May 6, 2026
From: NLX LLC
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 075567/0710 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2024
From: PAPANCEA, ANDREI; PAPANCEA, VLAD; SZERZO, PETER
To: NLX INC.
Reel/Frame 068393/0693 →
Continuity (1)
Provisional Application 63518267 · Aug 8, 2023
References Cited (64)
US 5892813A · Morin et al. · 1999 [cited by applicant]
US 7123709B1 · Montgomery et al. · 2006 [cited by applicant]
US 7406657B1 · Callaghan et al. · 2008 [cited by applicant]
US 7693720B2 · Kennewick et al. · 2010 [cited by applicant]
US 8046213B2 · Bakis et al. · 2011 [cited by applicant]
US 8195468B2 · Weider et al. · 2012 [cited by applicant]
US 8761379B2 · Davis et al. · 2014 [cited by applicant]
US 8983383B1 · Haskin · 2015 [cited by applicant]
US 9325845B2 · Nguyen et al. · 2016 [cited by applicant]
US 9521259B2 · Davis et al. · 2016 [cited by applicant]
US 9955205B2 · Covell et al. · 2018 [cited by applicant]
US 9961120B2 · Krishnan et al. · 2018 [cited by applicant]
US 10051131B2 · Saylor et al. · 2018 [cited by applicant]
US 10469664B2 · Pirat et al. · 2019 [cited by applicant]
US 10484539B2 · Kumar et al. · 2019 [cited by applicant]
US 10523816B1 · Neuer, III et al. · 2019 [cited by applicant]
US 10637989B1 · Josh et al. · 2020 [cited by applicant]
US 11276407B2 · Reshef et al. · 2022 [cited by applicant]
US 11461562B2 · Papancea · 2022 [cited by applicant]
US 11687737B2 · Papancea · 2023 [cited by applicant]
US 20020090062A1 · Mohamed Alhadad et al. · 2002 [cited by applicant]
US 20030162561A1 · Johnson et al. · 2003 [cited by applicant]
US 20030234763A1 · Hejza Litwiller · 2003 [cited by applicant]
US 20050010418A1 · McNair et al. · 2005 [cited by applicant]
US 20090149158A1 · Goldfarb et al. · 2009 [cited by applicant]
US 20090247136A1 · Srinivasan et al. · 2009 [cited by applicant]
US 20120069975A1 · Odinak et al. · 2012 [cited by applicant]
US 20130007147A1 · Toga et al. · 2013 [cited by applicant]
US 20130108994A1 · Srinivasa et al. · 2013 [cited by applicant]
US 20160171772A1 · Ryznar et al. · 2016 [cited by applicant]
US 20160219147A1 · Maxwell et al. · 2016 [cited by applicant]
US 20160301810A1 · Dimitroff et al. · 2016 [cited by applicant]
US 20170285848A1 · Rosenberg et al. · 2017 [cited by applicant]
US 20170324868A1 · Tamblyn et al. · 2017 [cited by applicant]
US 20170359463A1 · Segalis et al. · 2017 [cited by applicant]
US 20180052884A1 · Kale · 2018 [cited by examiner]
US 20190037077A1 · Konig et al. · 2019 [cited by applicant]
US 20190139432A1 · Jain et al. · 2019 [cited by applicant]
US 20190377544A1 · Darachi, Jr. et al. · 2019 [cited by applicant]
US 20200177730A1 · Dutta et al. · 2020 [cited by applicant]
US 20210084150A1 · Nagy · 2021 [cited by applicant]
US 20210124782A1 · Karaca · 2021 [cited by examiner]
US 20220164550A1 · Papancea et al. · 2022 [cited by applicant]
US 20230004731A1 · Papancea et al. · 2023 [cited by applicant]
US 20230065468A1 · Lu · 2023 [cited by examiner]
US 20230274103A1 · Papancea et al. · 2023 [cited by applicant]
US 20240311348A1 · Lutz · 2024 [cited by examiner]
WO WO2019236120A1 · 2019 [cited by applicant]
WO WO2022109149A1 · 2022 [cited by applicant]
CallVU, CallVU Visual IVR—Banking, [Online], Screenshots, Retrieved from the Internet: https://www.youtube.com/watch?v=94IIhLCNbZ0 , Sep. 2020, 2 pages. [cited by applicant]
ChoiceView, ChoiceView Visual IVR Insurance Claim [Online], Screenshots, Retrieved from the Internet: https://www.youtube.com/watch?v=FCFGZLp7qOE , Sep. 2020, 2 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2021/059911, mailed Mar. 1, 2022, 15 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/315,027 mailed Aug. 13, 2024, 28 pages. [cited by applicant]
Office Action for U.S. Appl. No. 17/532,662, mailed Apr. 4, 2022, 16 pages. [cited by applicant]
Office Action for U.S. Appl. No. 17/932,162, mailed Nov. 15, 2022, 9 pages. [cited by applicant]
Plum IVR Support, “How to Extend the Time the Caller Has to Enter Numbers,” [Online], Feb. 2010, Retrieved from the Internet: https://support.plumvoice.com/viewtopic.php?f=2&t=591 , 4 pages. [cited by applicant]
Uniphore, “Multimodal User Experience Design Best Practices,” [Online], Aug. 2021, Retrieved from the Internet: https://www.uniphore.com/blog/multimodal-user-experience-design-best-practices/ , 8 pages. [cited by applicant]
Uniphore, “What's a Multimodal Customer Experience?” [Online], Oct. 2019, Retrieved from the Internet: https://www.uniphore.com/blog/what-s-a-multimodal-customer-experience/ , 1 page. [cited by applicant]
Hu, Y. et al., “Code prompting: a neural symbolic method for complex reasoning in large language models,” arXiv:2305.18507v2 [cs.CL], Oct. 2023, 33 pages. [cited by applicant]
Huang, Q. et al., “PRODIGY: Enabling In-context Learning Over Graphs,” arXiv:2305.12600v1 [cs.LG], May 2023, 15 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2024/041520, mailed Oct. 30, 2024, 15 pages. [cited by applicant]
John, R. J. L. et al., “A natural language interface for dissemination of reproducible biomedical data science,” Med Image Comput Comput Assist Interv., Sep. 2018, vol. 11073, pp. 197-205. doi: 10.1007/978-3-030-00937-3… [cited by applicant]
Office Action for European Application No. 21835458.7 mailed Oct. 28, 2024, 10 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/315,027 mailed Nov. 12, 2024, 25 pages. [cited by applicant]
Cited By (4)
US 12,518,109 US 12,572,598 US 12,730,976 US 12,737,552