Multi-agent framework for natural language processing
The disclosure relates to a multi-agent framework that includes a plurality of language model (LM) agents that each perform a respective Natural Language Processing (NLP) task to analyze content having natural language text. An LM agent may execute a language model to perform its respective NLP task. For example, to identify target information within content, a first LM agent in the multi-agent framework may generate a summary of the content along with the target information, a second LM agent may extract, independently from the first LM agent, the target information and output reasoning that explains why the target information was extracted, and a third LM agent may verify that the target information was correctly identified based on the output of the first and second LM agents.
1 . A system, comprising:
a processor programmed to:
access a request to perform a natural language processing (NLP) task on content having natural language text;
generate, by a first language model (LM) agent in a multi-agent LM framework, a summarization output comprising a text summarization of the content based on a first NLP task specification and a first language model, wherein the first NLP task specification includes one or more instructions for the first LM agent to generate the text summarization and to include target information in the text summarization;
extract, by a second LM agent in the multi-agent framework, independently from the first LM agent, the target information from the content based on a second NLP task specification and execution of a second language model that is the same as or different from the first language model, wherein the second NLP task specification comprises one or more guidelines that encode how to identify the target information within the content;
generate, by the second LM agent, an extraction output comprising the target information extracted from the content;
execute, by a third LM agent in the multi-agent LM framework, a verification task to verify that the target information extracted by the second LM agent is correct based on a third NLP task specification that specifies how to perform the verification task, the first LM agent output and the second LM agent output; and
generate, based on execution of the verification task, a verification output comprising the target information or a replacement for the target information.
2 . The system of claim 1 , wherein the second NLP task specification further comprises an instruction to include a reasoning that explains why the target information was identified for extraction from the content, and wherein the second LM agent is configured to include the reasoning in the extraction output.
3 . The system of claim 2 , wherein the third LM agent uses the reasoning as part of the verification task.
4 . The system of claim 1 , wherein the first NLP task specification comprises a context that describes how to access the content and one or more guidelines that are to be adhered to during generation of the summarization output, and wherein the first LM agent is configured to use the context to generate the summarization output and the one or more guidelines to constrain the summarization output.
5 . The system of claim 1 , wherein the one or more guidelines that encode how to identify the target information within the content comprises one or more conditional statements that are evaluated against the content, and wherein the one or more language models executed by the second LM agent evaluates the one or more conditional statements to identify the target information within the content.
6 . The system of claim 5 , wherein the one or more conditional statements, when evaluated, results in identification of the target information from a first portion of the content instead of a second portion of the content.
7 . The system of claim 5 , wherein the one or more conditional statements, when evaluated, results in ignoring at least a portion of the content.
8 . The system of claim 1 , wherein to generate the verification output, the third LM agent is configured to execute a third language model that is different from or the same as the first language model.
9 . The system of claim 1 , wherein the third NLP task specification comprises one or more guidelines that are to be adhered to during generation of the verification output, and wherein the third LM agent is configured to use the one or more guidelines to analyze the summarization output and the extraction output and determine whether the target information from the extraction output is correct.
10 . The system of claim 1 , wherein the third LM agent is configured to determine, based on the one or more guidelines, that the target information is incorrect, and wherein the third LM agent is configured to identify the target information based on the one or more guidelines.
11 . A method, comprising:
accessing a request to perform a natural language processing (NLP) task on content having natural language text;
generating, by a first language model (LM) agent in a multi-agent LM framework, a summarization output comprising a text summarization of the content based on a first NLP task specification and a first language model, wherein the first NLP task specification includes one or more instructions for the first LM agent to generate the text summarization and to include target information in the text summarization;
extracting, by a second LM agent in the multi-agent framework, independently from the first LM agent, the target information from the content based on a second NLP task specification and execution of a second language model that is the same as or different from the first language model, wherein the second NLP task specification comprises one or more guidelines that encode how to identify the target information within the content;
generating, by the second LM agent, an extraction output comprising the target information extracted from the content;
executing, by a third LM agent in the multi-agent LM framework, a verification task to verify that the target information extracted by the second LM agent is correct based on a third NLP task specification that specifies how to perform the verification task, the first LM agent output and the second LM agent output; and
generating, based on execution of the verification task, a verification output comprising the target information or a replacement for the target information.
12 . The method of claim 11 , wherein the second NLP task specification further comprises an instruction to include a reasoning that explains why the target information was identified for extraction from the content, and wherein the second LM agent is configured to include the reasoning in the extraction output.
13 . The method of claim 12 , wherein the third LM agent uses the reasoning as part of the verification task.
14 . The method of claim 11 , wherein the first NLP task specification comprises a context that describes how to access the content and one or more guidelines that are to be adhered to during generation of the summarization output, the method further comprising: using, by the first LM agent, the context to generate the summarization output and the one or more guidelines to constrain the summarization output.
15 . The method of claim 11 , wherein the one or more guidelines that encode how to identify the target information within the content comprises one or more conditional statements that are evaluated against the content, and wherein the one or more language models executed by the second LM agent evaluates the one or more conditional statements to identify the target information within the content.
16 . The method of claim 15 , wherein the one or more conditional statements, when evaluated, results in identification of the target information from a first portion of the content instead of a second portion of the content.
17 . The method of claim 15 , wherein the one or more conditional statements, when evaluated, results in ignoring at least a portion of the content.
18 . The method of claim 11 , wherein generating the verification output, comprises executing, by the third LM agent, a third language model that is different from or the same as the first language model.
19 . The method of claim 11 , wherein the third NLP task specification comprises one or more guidelines that are to be adhered to during generation of the verification output, the method further comprising using, by the third LM agent, the one or more guidelines to analyze the summarization output and the extraction output and determine whether the target information from the extraction output is correct.
20 . A non-transitory computer readable medium storing instructions that, when executed by a processor, programs the processor to:
access a request to perform a natural language processing (NLP) task on content having natural language text;
generate, by a first language model (LM) agent in a multi-agent LM framework, a summarization output comprising a text summarization of the content based on a first NLP task specification and execution of a first language model, wherein the first NLP task specification includes one or more instructions for the first LM agent to generate the text summarization and to include target information in the text summarization;
identify, by a second LM agent in the multi-agent framework, independently from the first LM agent, the target information within the content based on a second NLP task specification and execution of a second language model that is the same as or different from the first language model, wherein the second NLP task specification includes one or more instructions for the second LM agent to identify and extract the target information from the content;
generate, by the second LM agent, an extraction output comprising the target information identified within the content; and
execute, by a third LM agent in the multi-agent LM framework, a verification task to validate the target information extracted by the second LM agent based on a third NLP task specification that specifies how to perform the verification task, the first LM agent output and the second LM agent output.