Agentic workflow for language model powered search and visualization on structured and unstructured data

1 . A method for answering a data query using a language model-based system having a plurality of nodes, the method comprising:

receiving the data query with an action planning node;

interpreting the data query to determine information to answer the data query;

determining an action plan for answering the data query in response to the information, wherein the action plan includes how to decompose the data query into sub-queries and actions to obtain the information for each of the sub-queries;

sending a call from the action planning node to an action node to execute a selected tool from a set of tools in response to the action plan, wherein the selected tool is configured to conduct a search of one or more structured data sources comprising an open subsurface data universe;

sending an output from the selected tool to the action planning node in response to the call;

sending the output from the action planning node to a final node;

synthesizing the output, via the final node, to form a response to the data query; and

sending the response to the data query from the final node to a gate node, wherein the gate node is configured to access a language model to:

determine that the response successfully answers the data query; and

send a final answer based on the response to an end user in response to determining that the response successfully answers the data query.

2 . The method of claim 1 , wherein the selected tool includes a workflow configured to retrieve data from one or more databases or documents, transform the data, run domain interpretation workflows based upon the data, or generate charts based upon the data, and wherein the workflow being configured to retrieve data from one or more databases includes retrieving the data from at least one of a relational database, a non-relational database, a graph database, or a vector database.

3 . The method of claim 1 , wherein the set of tools include a document search tool configured to conduct a search of one or more unstructured data sources, a document summarization tool configured to summarize information within a document, a structured search and summarization tool configured to conduct a search of the one or more structured data sources and summarize search results from the search, a get related data tool configured to determine parent and child entities for a specific record, a plotting tool configured to generate charts from results retrieved from at least one of the search of the one or more structured data sources and the search of the one or more unstructured data sources, and a packaging tool configured to package the search results from the search of the one or more structured data sources.

4 . The method of claim 1 , wherein the action planning node includes, or is configured to access, at least one of the language model or a domain store of domain centric information, wherein the domain centric information includes at least one of ontological data, knowledge graphs, or vector databases.

5 . The method of claim 1 , further comprising:

determining, by the action planning node, whether additional information is needed to answer the data query;

in response to determining that additional information is needed, sending one or more additional calls, by the action planning node, to the action node to execute one or more selected tools from the set of tools and sending, by the action node, output from the one or more selected tools to the action planning node.

6 . The method of claim 1 , further comprising

in response to determining that the response has not successfully answered the data query, sending a call, by the gate node, back to the action planning node with intermediate steps of reasoning to improvise the action plan comprising query decomposition and tool calling and gather different or additional data.

7 . The method of claim 6 , wherein the intermediate steps in reasoning include at least one of a sequence of tools calls, one or more subqueries from query decomposition, or one or more input arguments to one or more tools from the set of tools.

8 . The method of claim 6 , further comprising:

receiving, by an agent summarizer node, the intermediate steps in reasoning;

determining, by the agent summarizer node, one or more insights from the intermediate steps of reasoning;

storing the one or more insights into an insights store; and

enhancing the action planning node by synthesizing examples and instructions using the one or more insights from the insights store.

9 . The method of claim 8 , wherein the one or more insights includes one or more of: outcomes from tool calls, query decomposition, or tool input arguments.

10 . A computing system, comprising:

one or more processors; and

a memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations comprising:

receiving a data query with an action planning node, wherein the data query includes at least one of a natural language query, a conversational history, or user context;

interpreting the data query to determine information to answer the data query;

determining an action plan for answering the data query in response to the information, wherein the action plan includes how to decompose the data query into sub-queries and actions to obtain the information for each of the sub-queries;

sending a call from the action planning node to an action node to execute a selected tool from a set of tools in response to the action plan, wherein the selected tool includes a workflow configured to retrieve data from one or more databases or documents, transform the data, run domain interpretation workflows based upon the data, and/or generate charts based upon the data, and wherein the selected tool is configured to conduct a search of one or more structured data sources comprising an open subsurface data universe;

sending an output from the selected tool to the action planning node in response to the call;

sending the output from the action planning node to a final node;

synthesizing the output, via the final node, to form a response to the data query;

sending the response to the data query from the final node to a gate node, wherein the gate node includes, or is configured to access, a language model to:

determine that the response successfully answers the data query; and

send a final answer based on the response to an end user in response to determining that the response successfully answers the data query; and

in response to sending the final answer based on the response to the end user, generating one or more signals, via the computing system, to instruct a physical action to occur at a wellbore, wherein the physical action comprises selecting where to drill a wellbore, drilling the wellbore, varying a weight and/or torque on a drill bit that is drilling the wellbore, varying a drilling trajectory of the wellbore, or varying a concentration and/or flow rate of a fluid pumped into the wellbore.

11 . The computing system of claim 10 , wherein the operations further comprise sending a call, by the gate node, back to the action planning node with intermediate steps of reasoning to improvise the action plan comprising query decomposition and tool calling and gather different or additional data.

12 . The computing system of claim 11 , wherein the operations further comprise:

receiving, by an agent summarizer node, the intermediate steps of reasoning;

determining, by the agent summarizer node, one or more insights from the intermediate steps of reasoning;

storing the one or more insights into an insights store; and

enhancing the action planning node by synthesizing examples and instructions using the one or more insights from the insights store.

13 . The computing system of claim 12 , wherein the one or more insights includes one or more of: outcomes from tool calls, query decomposition, or tool input arguments.

14 . The computing system of claim 13 , wherein the intermediate steps in reasoning include at least one of a sequence of tools calls, one or more subqueries from query decomposition, or one or more input arguments to one or more tools from the set of tools.

15 . The computing system of claim 10 , wherein the set of tools include one or more of a document search tool configured to conduct a search of one or more unstructured data sources, a document summarization tool configured to summarize information within a document, a structured search and summarization tool configured to conduct a search of a structured data repository and summarize search results from the search, a get related data tool configured to determine parent and child entities for a specific record, a plotting tool configured to generate charts from results retrieved from at least one of the search of the structured data repository and the search of the one or more unstructured data sources, or a packaging tool configured to package the search results from the search of the structured data repository.

16 . The computing system of claim 10 , wherein the action planning node includes, or can access, at least one of the language model or a domain store of domain centric information, wherein the domain centric information includes at least one of ontological data, knowledge graphs, or vector databases.

17 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations, the operations comprising:

receiving a data query with an action planning node, wherein the data query includes at least one of a natural language query, a conversational history, or user context;

interpreting the data query to determine information to answer the data query;

determining an action plan for answering the data query in response to the information, wherein the action plan includes how to decompose the data query into sub-queries and actions to obtain the information for each of the sub-queries;

sending a call from the action planning node to an action node to execute a selected tool from a set of tools in response to the action plan, wherein the selected tool includes a workflow configured to retrieve data from one or more databases or documents, transform the data, run domain interpretation workflows based upon the data, and/or generate charts based upon the data;

sending an output from the selected tool to the action planning node in response to the call;

synthesizing the output to form a response to the data query with a final node;

sending the response to the data query to a gate node;

determining, by the gate node, that the response does not successfully answer the data query, wherein the gate node includes, or is configured to access, a language model; and

sending a second call, by the gate node, back to the action planning node with intermediate steps of reasoning to improvise the action plan comprising query decomposition and tool calling and gather different or additional data, wherein the intermediate steps of reasoning comprise sending one or more additional calls to the action planning node for the action planning node to execute while determining a new action plan for answering the data query.

18 . The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise:

receiving, by an agent summarizer node, the intermediate steps of reasoning;

determining, by the agent summarizer node, one or more insights from the intermediate steps of reasoning;

storing the one or more insights into an insights store; and

enhancing the action planning node by synthesizing examples and instructions using the one or more insights from the insights store.

19 . The non-transitory computer-readable medium of claim 18 , wherein the one or more insights includes one or more of: outcomes from tool calls, query decomposition, or tool input arguments.

20 . The non-transitory computer-readable medium of claim 17 , wherein the workflow configured to retrieve data from one or more databases includes retrieving data from at least one of a relational database, a non-relational database, a graph database, or a vector database.