Method and system for information extraction and aggregation

1 . A method for using large language models (LLMs) to extract and aggregate information in order to generate a structured response to a user query, the method being implemented by at least one processor, the method comprising:

receiving a query from a user;

using a first LLM to act as a query replication LLM agent to transform the query into a plurality of action items;

using a second LLM to act as a plan generator LLM agent to generate at least one plan for responding to each of the plurality of action items;

using a third LLM to act as a plan selection LLM agent to select an optimum plan from among the generated at least one plan for executing each of the plurality of action items;

using a fourth LLM to act as a plan orchestrator LLM agent to assemble a plurality of fifth LLMs to act as expert LLM sub-agents to perform respective tasks based on the selected optimum plan;

obtaining, from each of the expert LLM sub-agents, a respective output that results from a performance of the corresponding task; and

using the plan orchestrator LLM agent to aggregate the respective outputs into a single response to the query,

wherein the method further comprises using the plan orchestrator LLM agent to notify the user of a progress status each time the respective output that results from the performance of the corresponding task when the respective output is obtained from each of the expert LLM sub-agents, wherein each of the expert LLM sub-agents do not interact with each other, and wherein when a particular one of the expert LLM sub-agents is unable to successfully execute the corresponding one of the respective tasks, the progress status includes information that relates to the inability of the particular one of the expert LLM sub-agents to successfully execute the corresponding one of the respective tasks.

2 . The method of claim 1 , wherein each expert LLM sub-agent is configured to generate the respective output by extracting information from a plurality of publicly available governmental regulatory filings.

3 . The method of claim 2 , wherein the plurality of publicly available governmental regulatory filings includes filings that relate to securities transactions.

4 . The method of claim 1 , wherein the query replication LLM agent is configured to transform the query into the plurality of action items by spelling out acronyms from within the query, determining a time frame for which the query is applicable, and rewording the query based on an input expectation associated with the planning generator LLM agent.

5 . The method of claim 1 , wherein the plan generator LLM agent is configured to generate the at least one plan for responding to the plurality of action items by decomposing the plurality of action items into the respective tasks that are suitable for performance by at least one of the expert LLM sub-agents.

6 . The method of claim 1 , wherein each expert LLM sub-agent is configured to operate independently with respect to the other expert LLM sub-agents.

7 . The method of claim 1 , wherein the plan orchestrator LLM agent is configured to determine, for each respective output obtained from one of the expert LLM sub-agents, whether the respective output includes at least one from among an error and a hallucinated result.

8 . The method of claim 7 , wherein the plan orchestrator LLM agent is further configured to determine whether any of the respective tasks requires a reperformance.

9 . A computing apparatus for using large language models (LLMs) to extract and aggregate information in order to generate a structured response to a user query, the computing apparatus comprising:

a processor;

a memory; and

a communication interface coupled to each of the processor and the memory,

wherein the processor is configured to:

receive, from a user via the communication interface, a query;

use a first LLM to act as a query replication LLM agent to transform the query into a plurality of action items;

use a second LLM to act as a plan generator LLM agent to generate at least one plan for responding to each of the plurality of action items;

use a third LLM to act as a plan selection LLM agent to select an optimum plan from among the generated at least one plan for executing each of the plurality of action items;

use a fourth LLM to act as a plan orchestrator LLM agent to assemble a plurality of fifth LLMs to act as expert LLM sub-agents to perform respective tasks based on the selected optimum plan;

obtain, from each of the expert LLM sub-agents, a respective output that results from a performance of the corresponding task; and

use the plan orchestrator LLM agent to aggregate the respective outputs into a single response to the query,

wherein the processor is further configured to use the plan orchestrator LLM agent to notify the user of a progress status each time the respective output that results from the performance of the corresponding task is obtained from each of the expert LLM sub-agents, wherein each of the expert LLM sub-agents do not interact with each other, and wherein when a particular one of the expert LLM sub-agents is unable to successfully execute the corresponding one of the respective tasks, the progress status includes information that relates to the inability of the particular one of the expert LLM sub-agents to successfully execute the corresponding one of the respective tasks.

10 . The computing apparatus of claim 9 , wherein each expert LLM sub-agent is configured to generate the respective output by extracting information from a plurality of publicly available governmental regulatory filings.

11 . The computing apparatus of claim 10 , wherein the plurality of publicly available governmental regulatory filings includes filings that relate to securities transactions.

12 . The computing apparatus of claim 9 , wherein the query replication LLM agent is configured to transform the query into the plurality of action items by spelling out acronyms from within the query, determining a time frame for which the query is applicable, and rewording the query based on an input expectation associated with the planning generator LLM agent.

13 . The computing apparatus of claim 9 , wherein the plan generator LLM agent is configured to generate the at least one plan for responding to the plurality of action items by decomposing the plurality of action items into the respective tasks that are suitable for performance by at least one of the expert LLM sub-agents.

14 . The computing apparatus of claim 9 , wherein each expert LLM sub-agent is configured to operate independently with respect to the other expert LLM sub-agents.

15 . The computing apparatus of claim 9 , wherein the plan orchestrator LLM agent is configured to determine, for each respective output obtained from one of the expert LLM sub-agents, whether the respective output includes at least one from among an error and a hallucinated result.

16 . The computing apparatus of claim 15 , wherein the plan orchestrator LLM agent is further configured to determine whether any of the respective tasks requires a reperformance.

17 . A non-transitory computer readable storage medium storing instructions for using large language models (LLMs) to extract and aggregate information in order to generate a structured response to a user query, the storage medium comprising executable code which, when executed by a processor, causes the processor to:

receive a query from a user;

use a first LLM to act as a query replication LLM agent to transform the query into a plurality of action items;

use a second LLM to act as a plan generator LLM agent to generate at least one plan for responding to each of the plurality of action items;

use a third LLM to act as a plan selection LLM agent to select an optimum plan from among the generated at least one plan for executing each of the plurality of action items;

use a fourth LLM to act as a plan orchestrator LLM agent to assemble a plurality of fifth LLMs to act as expert LLM sub-agents to perform respective tasks based on the selected optimum plan;

obtain, from each of the expert LLM sub-agents, a respective output that results from a performance of the corresponding task; and

use the plan orchestrator LLM agent to aggregate the respective outputs into a single response to the query,

wherein when executed, the executable code further causes the processor to use the plan orchestrator LLM agent to notify the user of a progress status each time the respective output that results from the performance of the corresponding task is obtained from each of the expert LLM sub-agents, wherein each of the expert LLM sub-agents do not interact with each other, and wherein when a particular one of the expert LLM sub-agents is unable to successfully execute the corresponding one of the respective tasks, the progress status includes information that relates to the inability of the particular one of the expert LLM sub-agents to successfully execute the corresponding one of the respective tasks.

18 . The storage medium of claim 17 , wherein each expert LLM sub-agent is configured to generate the respective output by extracting information from a plurality of publicly available governmental regulatory filings.