Multiparty communication using a large language model intermediary

1. A method for providing multiparty communication using a large language model (LLM), the method being implemented by a computer system, the method comprising:

receiving input messages from user devices associated with a plurality of communication parties participating in a scenario, the LLM being configured to understand the scenario;

generating a text input prompt for the LLM based on a scenario template, wherein the scenario template includes rules to:

organize the input messages based on the scenario, wherein the plurality of communication parties and the LLM are assigned roles as participants in the scenario; and

instruct the LLM to analyze text input prompts and compute response outputs of the LLM as defined by an assigned role of the LLM in the scenario, the text input prompts including the text input prompt;

sending the text input prompt to the LLM, the LLM analyzing the text input prompt to compose a response output of the LLM;

analyzing the response output of the LLM, based on a syntax structure provided in the scenario template for the LLM, to generate output messages to be sent to one or more communication parties of the plurality of communication parties, the one or more communication parties being identified by a role of the one or more communication parties in the response output of the LLM; and

sending the output messages to the user devices associated with the one or more communication parties of the plurality of communication parties from a communication endpoint representing the LLM in the assigned role of the LLM in the scenario.

2. The method of claim 1 , wherein the scenario template includes:

an activity in which the LLM and the plurality of communication parties are to be participating;

a number of communication parties of the plurality of communication parties;

a reference to the scenario, the scenario providing a context for the activity, wherein the LLM is configured to understand the scenario based on one of the following: pre-training of the LLM, fine-tuning of the LLM with additional training data, and by incorporating additional data into the text input prompt using Retrieval Augmented Generation;

the roles of each of the plurality of communication parties assigned based on the scenario;

the assigned role of the LLM assigned based on the scenario and the activity;

a first syntax structure used to structure the input messages from the plurality of communication parties and assign the input messages based on the roles in the scenario; and

a second syntax structure used by the LLM to structure the response output to identify a content and desired recipients of the output messages to be sent on behalf of the LLM.

3. The method of claim 1 , further comprising repeating operations in a further cycle each time a further input message is received from the one or more communication parties of the plurality of communication parties with prior messages, the prior messages including messages from the LLM, included as the input messages in a further text input prompt following the scenario template.

4. The method of claim 3 , wherein the repeating the operations in the further cycle is triggered based on one or more of the following: an algorithm, an external data source, and the LLM analyzing the input messages to determine if the LLM is to engage in the further cycle.

5. The method of claim 1 , wherein the input messages and other elements of the text input prompt originate from sources, the sources including one or more of the following: data events, machines, sensors, further bots, and further sources accessible to a communication party of the plurality of communication parties and enterprises associated with the plurality of communication parties.

6. The method of claim 1 , wherein the input messages are associated with one or more modalities, the one or more modalities including one of the following: text, an image, audio, video, and a location;

wherein the LLM is configured to compose the response output having the one or more modalities associated with the input messages.

7. The method of claim 1 , wherein

the scenario template includes further rules on an action to be taken by the LLM when the LLM is requested to provide information that is not included in the text input prompt;

the further rules include one or more of the following:

generating a message to request the information from a communication party of the plurality of communication parties, wherein the communication party of the plurality of communication parties is expected, based on a role of the communication party of the plurality of communication parties or the rules in the scenario template, to have the information; and

generating a further message to request the information from an external data source, the external data source including one or more of the following: further machines, sensors, bots, and further sources including systems, databases and other sources accessible to the communication party of the plurality of communication parties or enterprises associated with the communication party of the plurality of communication parties, wherein the communication party of the plurality of communication parties is expected, based on the role of the communication party of the plurality of communication parties or the rules in the scenario template, to have the information.

8. The method of claim 1 , wherein the LLM is configured to acknowledge a receipt of a message from a communication party of the plurality of communication parties.

9. The method of claim 1 , wherein the LLM is configured to:

analyze prior messages, the prior messages including messages from the LLM;

determine when follow-up by one or more of the plurality of communication parties is expected; and

cause a notification to be sent to the respective communication party.

10. The method of claim 1 , wherein the LLM is configured to:

recognize that a subset of the plurality of communication parties is communicating with each other directly; and

in response to the recognition, refrain from intervening in the multiparty communication between the subset of the plurality of communication parties.

11. The method of claim 1 , wherein the LLM is configured to perform one or more of the following:

send instructions and messages to one or more external systems;

perform an action in the one or more external systems;

trigger a workflow in the one or more external systems;

ascertain that the LLM has gathered necessary input to perform the action or trigger the workflow;

estimate, based on an analysis of a conversation associated with the multiparty communication, a certainty that the action is to be taken;

evaluate the certainty against a threshold set by an organization associated with the one or more external systems;

based on the evaluation that the certainty is above the threshold, execute the action or trigger the workflow based on the necessary input gathered from the multiparty communication; and

when the certainty is below the threshold, send a message to an appropriate party of the plurality of communication parties for a confirmation or an instruction about how to proceed.

12. The method of claim 1 , wherein the LLM is configured to:

generate a code in a computer language associated with one or more external systems; and

based on the code, perform or more of the following:

send instructions to the one or more external systems;

send messages to the one or more external systems;

perform an action in the one or more external systems; and

trigger a workflow in the one or more external systems.

13. The method of claim 1 , wherein the LLM is configured to:

understand a language of the one or more communication parties of the plurality of communication parties;

communicate with the one or more communication parties in the language chosen by the one or more communication parties in the multiparty communication; and

automatically translate messages of a first communication party of the one or more communication parties from a first language chosen by the first communication party to a second language chosen by a second communication party of the one or more communication parties.

14. The method of claim 1 , wherein the LLM is configured to orchestrate a plurality of dialogues of one or more subsets of the plurality of communication parties as part of the same conversation associated with the multiparty communication.

15. The method of claim 1 , further comprising instructing the LLM, in the text input prompt, to perform one or more of the following:

add a communication party to the plurality of communication parties based on one or more of the following: a milestone in the scenario, an external event, a request from a further communication party, a requirement for information that is not available in the text input prompt or among the plurality of communication parties, and a demand provided in the scenario and inferred by the LLM;

based on the text input prompt, send messages to systems in control of the plurality of communication parties such that the communication party is invited to join the plurality of communication parties;

drop the communication party of the plurality of communication parties based on one or more of the following: the milestone in the scenario, the external event, the request from the further communication party, a request from the communication party to be dropped, and the demand provided in the scenario and inferred by the LLM; and,

based on the text input prompt, send further messages to systems in control of the plurality of communication parties such that the communication party is dropped from the plurality of communication parties.

16. A system for providing multiparty communication using a large language model (LLM), the system comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the processor to:

receive input messages from user devices associated with a plurality of communication parties participating in a scenario, the LLM being configured to understand the scenario;

generate a text input prompt for the LLM based on a scenario template, wherein the scenario template includes rules to:

organize the input messages based on the scenario, wherein the plurality of communication parties and the LLM are assigned roles as participants in the scenario; and

instruct the LLM to analyze text input prompts and compute response outputs of the LLM as defined by an assigned role of the LLM in the scenario, the text input prompts including the text input prompt;

send the text input prompt to the LLM, the LLM analyzing the text input prompt to compose a response output of the LLM;

analyze the response output of the LLM, based on a syntax structure provided in the scenario template for the LLM, to generate output messages to be sent to one or more communication parties of the plurality of communication parties, the one or more communication parties being identified by a role of the one or more communication parties in the response output of the LLM; and

send the output messages to the user devices associated with the one or more communication parties of the plurality of communication parties from a communication endpoint representing the LLM in the assigned role of the LLM in the scenario.

17. The system of claim 16 , wherein the scenario template includes:

an activity in which the LLM and the plurality of communication parties are to be participating;

a number of communication parties of the plurality of communication parties;

a reference to the scenario, the scenario providing a context for the activity, wherein the LLM is configured to understand the scenario based on one of the following: pre-training of the LLM, fine-tuning of the LLM with additional training data, and by incorporating additional data into the text input prompt using Retrieval Augmented Generation;

the roles of each of the plurality of communication parties assigned based on the scenario;

the assigned role of the LLM assigned based on the scenario and the activity;

a first syntax structure used to structure the input messages from the plurality of communication parties and assign the input messages based on the roles in the scenario; and

a second syntax structure used by the LLM to structure the response output to identify a content and desired recipients of the output messages to be sent on behalf of the LLM.

18. The system of claim 16 , wherein the processor is further configured to repeat operations in a further cycle each time a further input message is received from any of the plurality of communication parties with prior messages, the prior messages including messages from the LLM included as the input messages in a further text input prompt following the scenario template.

19. The system of claim 18 , wherein the repeating the operations in the further cycle is triggered based on one or more of the following: an algorithm, an external data source, and the LLM analyzing the input messages to determine if the LLM is to engage in the further cycle.

20. The system of claim 16 , wherein the input messages and other elements of the text input prompt originate from sources, the sources including one or more of the following: data events, machines, sensors, further bots, and further sources accessible to a communication party of the plurality of communication parties and enterprises associated with the plurality of communication parties.

21. The system of claim 16 , wherein the input messages are associated with one or more modalities, the one or more modalities including one of the following: text, an image, audio, video, and a location;

wherein the LLM is configured to compose the response output having the one or more modalities associated with the input messages.

22. The system of claim 16 , wherein the scenario template includes further rules on an action to be taken by the LLM when the LLM is requested to provide information that is not included in the text input prompt and wherein the further rules include one or more of the following:

generating a message to request the information from a communication party of the plurality of communication parties, wherein the communication party of the plurality of communication parties is expected, based on a role of the communication party of the plurality of communication parties or the rules in the scenario template, to have the information; and

generating a further message to request the information from an external data source, the external data source including one or more of the following: further machines, sensors, bots, and further sources including systems, databases and other sources accessible to the communication party of the plurality of communication parties or enterprises associated with the communication party of the plurality of communication parties, wherein the communication party of the plurality of communication parties is expected, based on the role of the communication party of the plurality of communication parties or the rules in the scenario template, to have the information.

23. The system of claim 16 , wherein the LLM is configured to:

analyze prior messages, the prior messages including messages from the LLM;

determine when follow-up by one or more of the plurality of communication parties is expected; and

cause a notification to be sent to the respective communication party.

24. The system of claim 16 , wherein the LLM is configured to:

recognize that a subset of the plurality of communication parties is communicating with each other directly; and

in response to the recognition, refrain from intervening in the multiparty communication between the subset of the plurality of communication parties.

25. The system of claim 16 , wherein the LLM is configured to perform one or more of the following:

send further instructions and messages to one or more external systems;

perform an action in the one or more external systems;

trigger a workflow in the one or more external systems;

ascertain that the LLM has gathered necessary input to perform the action or trigger the workflow;

estimate, based on an analysis of a conversation associated with the multiparty communication, a certainty that the action is to be taken;

evaluate the certainty against a threshold set by an organization associated with the one or more external systems;

based on the evaluation that the certainty is above the threshold, execute the action or trigger the workflow based on the necessary input gathered from the multiparty communication; and

when the certainty is below the threshold, send a message to an appropriate party of the plurality of communication parties for a confirmation or an instruction about how to proceed.

26. The system of claim 16 , wherein the LLM is configured to:

generate a code in a computer language associated with one or more external systems; and

based on the code, perform or more of the following:

send further instructions to the one or more external systems;

send messages to the one or more external systems;

perform an action in the one or more external systems; and

trigger a workflow in the one or more external systems.

27. The system of claim 16 , wherein the LLM is configured to:

understand a language of the one or more communication parties of the plurality of communication parties;

communicate with the one or more communication parties in the language chosen by the one or more communication parties in the multiparty communication; and

automatically translate messages of a first communication party of the one or more communication parties from a first language chosen by the first communication party to a second language chosen by a second communication party of the one or more communication parties.

28. The system of claim 16 , wherein the LLM is configured to orchestrate a plurality of dialogues of one or more subsets of the plurality of communication parties as part of the same conversation associated with the multiparty communication.

29. The system of claim 19 , wherein the processor is further configured to instruct the LLM, in the text input prompt, to perform one or more of the following:

add a communication party to the plurality of communication parties based on one or more of the following: a milestone in the scenario, an external event, a request from a further communication party, a requirement for information that is not available in the text input prompt or among the plurality of communication parties, and a demand provided in the scenario and inferred by the LLM;

based on the text input prompt, send messages to systems in control of the plurality of communication parties such that the communication party is invited to join the plurality of communication parties;

drop the communication party of the plurality of communication parties based on one or more of the following: the milestone in the scenario, the external event, the request from the further communication party, a request from the communication party to be dropped, and the demand provided in the scenario and inferred by the LLM; and

based on the text input prompt, send further messages to the systems in control of the plurality of communication parties such that the communication party is dropped from the plurality of communication parties.

30. A method for providing multiparty communication using a large language model (LLM), the method being implemented by a computer system, the method comprising:

receiving input messages from user devices associated with a plurality of communication parties participating in a scenario, the LLM being configured to understand the scenario;

generating a text input prompt for the LLM, the text input prompt being generated based on a scenario template, wherein the scenario template includes rules to:

organize the input messages based on the scenario, wherein the plurality of communication parties and the LLM are assigned roles as participants in the scenario; and

instruct the LLM to analyze text input prompts and compute response outputs of the LLM as defined by an assigned role of the LLM in the scenario, the text input prompts including the text input prompt;

wherein the scenario template includes:

an activity in which the LLM and the plurality of communication parties are to be participating;

a number of communication parties of the plurality of communication parties;

a reference to the scenario, the scenario providing a context for the activity, wherein the LLM is configured to understand the scenario based on one of the following: pre-training of the LLM, fine-tuning of the LLM with additional training data, and by incorporating additional data into the text input prompt using Retrieval Augmented Generation;

the roles of each of the plurality of communication parties assigned based on the scenario;

the assigned role of the LLM assigned based on the scenario and the activity;

a first syntax structure used to structure the input messages from the plurality of communication parties and assign the input messages based on the roles in the scenario;

a second syntax structure used by the LLM to structure a response output of the LLM to identify a content and desired recipients of the output messages to be sent on behalf of the LLM; and

further rules on an action to be taken by the LLM when the LLM is requested to provide information that is not included in the text input prompt, wherein the further rules include generating a message to request the information from a communication party of the plurality of communication parties, wherein the communication party of the plurality of communication parties is expected, based on a role of the communication party of the plurality of communication parties or the rules in the scenario template, to have the information;

sending the text input prompt to the LLM, the LLM analyzing the text input prompt to compose the response output of the LLM, the response output including one or more messages to be sent to one or more communication parties of the plurality of communication parties and multiparty communication instructions, wherein the LLM is configured to:

recognize that a subset of the plurality of communication parties is communicating with each other directly;

based on the recognition, refrain from intervening in the multiparty communication between the subset of the plurality of communication parties; and

orchestrate a plurality of dialogues of one or more subsets of the plurality of communication parties as a part of the same conversation associated with the multiparty communication;

analyzing the response output of the LLM, based on a syntax structure provided in the scenario template for the LLM, to generate, based on the one or more messages, output messages to be sent to the one or more communication parties of the plurality of communication parties, the one or more communication parties being identified by a role of the one or more communication parties in the response output of the LLM;

based on the multiparty communication instructions, adding a tag to at least one output message of the output messages;

based on the multiparty communication instructions, sending the output messages to the user devices associated with the one or more communication parties of the plurality of communication parties from a communication endpoint representing the LLM in the assigned role of the LLM in the scenario;

analyzing prior messages, the prior messages including messages from the LLM, to determine when an attention of one or more of the plurality of communication parties is required and cause a notification to be sent to the one or more of the plurality of communication parties; and

repeating operations in a further cycle each time a further input message is received from the one or more communication parties of the plurality of communication parties with the prior messages, the prior messages being included as the input messages in a further text input prompt following the scenario template.