Granted Patent B1

US 12,340,181 · App. 18/933,483 · Granted Jun 24, 2025

Inventors: Jia You (Cary, NC); Tieyi Guo (Frisco, TX); Byung Chun (Kingston, MA); Brian Christoper Mansfield (Milford, OH)

Assignee: FMR LLC

G06F40/35H04L51/02

Abstract

Claims (46)

1. A computer system for conversation dialogue orchestration in virtual assistant communication sessions, the system comprising a server computing device having a memory that stores computer-executable instructions and a processor that executes the computer-executable instructions to:

establish chat-based communication sessions between a virtual assistant application of the server computing device and each of a plurality of client computing devices;

for each of the chat-based communication sessions:

capture, by the virtual assistant application, an utterance generated by a user of the client computing device during the chat-based communication session;

process, by the virtual assistant application, the utterance to instantiate a dialogue behavior tree comprising a plurality of workflow agents each associated with executable code for completing a corresponding workflow action, the plurality of workflow agents including an authentication agent for verifying an identify of a user at the client computing device;

traverse, by the virtual assistant application, the dialogue behavior tree to generate a response to the utterance, including for each workflow agent:

evaluating one or more conditions associated with the workflow agent to determine whether to execute the code in the workflow agent,

when the conditions associated with the workflow agent are met, executing the code in the workflow agent to complete the workflow action and storing a sub-response associated with the completed workflow action in a dialogue memory comprising an in-memory data store, and

storing a bit vector in the dialogue memory, the bit vector comprising a plurality of binary values representing the determination of whether to execute the code in each workflow agent made by the virtual assistant application during traversal of the dialogue behavior tree,

coalesce, by the virtual assistant application, the sub-responses stored in the dialogue memory to generate a final response to the utterance; and

transmit, by the virtual assistant application, the final response to the client computing device for display during the chat-based communication session;

aggregate the bit vectors generated during each of the chat-based communication sessions to generate a real-time view of user interactions with the dialogue behavior tree; and

scale one or more computing resources of an authentication server cluster that hosts the authentication agent when the aggregated bit vectors indicate that a minimum number of chat-based communication sessions invoke the authentication agent.

2. The system of claim 1 , wherein processing the utterance to instantiate a dialogue behavior tree comprises rewriting the utterance using a first natural language understanding (NLU) algorithm.

3. The system of claim 2 , wherein the virtual assistant application:

creates one or more sub-utterances using the rewritten utterance; and

extracts an objective comprising one or more of: a topic, a subtopic, a local objective, a dialog act, and an entity from each of the sub-utterances using a second NLU algorithm.

4. The system of claim 3 , wherein the virtual assistant application analyzes the extracted objectives to determine a context used to instantiate the dialogue behavior tree.

5. The system of claim 1 , wherein the virtual assistant application evaluates the one or more conditions associated with a plurality of different workflow agents in parallel during traversal of the dialogue behavior tree.

6. The system of claim 5 , wherein the virtual assistant application executes the code in a plurality of different workflow agents in parallel during traversal of the dialogue behavior tree.

7. The system of claim 1 , wherein one or more of the plurality of workflow agents in the dialogue behavior tree comprises a behavior sub-tree including a plurality of workflow sub-agents each associated with executable code for completing a corresponding workflow action.

8. The system of claim 1 , wherein evaluating one or more conditions associated with the workflow agent to determine whether to execute the code in the workflow agent comprises comparing one or more session values for the chat-based communication session against the one or more conditions associated with the workflow agent to determine whether the conditions are met.

9. The system of claim 8 , wherein the executable code for completing a corresponding workflow action comprises a function call that invokes one or more application resources coupled to the server computing device.

10. A computerized method of conversation dialogue orchestration in virtual assistant communication sessions, the method comprising:

establishing, by a server computing device, chat-based communication sessions between a virtual assistant application of the server computing device and each of a plurality of client computing devices;

for each of the plurality of chat-based communication sessions:

capturing, by the virtual assistant application, an utterance generated by a user of the client computing device during the chat-based communication session;

processing, by the virtual assistant application, the utterance to instantiate a dialogue behavior tree comprising a plurality of workflow agents each associated with executable code for completing a corresponding workflow action, the plurality of workflow agents including an authentication agent for verifying an identify of a user at the client computing device;

traversing, by the virtual assistant application, the dialogue behavior tree to generate a response to the utterance, including for each workflow agent:

evaluating one or more conditions associated with the workflow agent to determine whether to execute the code in the workflow agent, and

when the conditions associated with the workflow agent are met, executing the code in the workflow agent to complete the workflow action and storing a sub-response associated with the completed workflow action in a dialogue memory comprising an in-memory data store, and

storing a bit vector in the dialogue memory, the bit vector comprising a plurality of binary values representing the determination of whether to execute the code in each workflow agent made by the virtual assistant application during traversal of the dialogue behavior tree,

coalescing, by the virtual assistant application, the sub-responses stored in the dialogue memory to generate a final response to the utterance; and

transmitting, by the virtual assistant application, the final response to the client computing device for display during the chat-based communication session;

aggregating, by the server computing device, the bit vectors generated during each of the chat-based communication sessions to generate a real-time view of user interactions with the dialogue behavior tree; and

scaling, by the server computing device, one or more computing resources of an authentication server cluster that hosts the authentication agent when the aggregated bit vectors indicate that a minimum number of chat-based communication sessions invoke the authentication agent.

11. The method of claim 10 , wherein processing the utterance to instantiate a dialogue behavior tree comprises rewriting the utterance using a first natural language understanding (NLU) algorithm.

12. The method of claim 11 , further comprising:

creating, by the virtual assistant application, one or more sub-utterances using the rewritten utterance; and

extracting, by the virtual assistant application, an objective comprising one or more of: a topic, a subtopic, a local objective, a dialog act, and an entity from each of the sub-utterances using a second NLU algorithm.

13. The method of claim 12 , further comprising analyzing, by the virtual assistant application, the extracted objectives to determine a context used to instantiate the dialogue behavior tree.

14. The method of claim 10 , further comprising evaluating, by the virtual assistant application, the one or more conditions associated with a plurality of different workflow agents in parallel during traversal of the dialogue behavior tree.

15. The system of claim 14 , further comprising executing, by the virtual assistant application, the code in a plurality of different workflow agents in parallel during traversal of the dialogue behavior tree.

16. The method of claim 10 , wherein one or more of the plurality of workflow agents in the dialogue behavior tree comprises a behavior sub-tree including a plurality of workflow sub-agents each associated with executable code for completing a corresponding workflow action.

17. The method of claim 10 , wherein evaluating one or more conditions associated with the workflow agent to determine whether to execute the code in the workflow agent comprises comparing one or more session values for the chat-based communication session against the one or more conditions associated with the workflow agent to determine whether the conditions are met.

18. The method of claim 17 , wherein the executable code for completing a corresponding workflow action comprises a function call that invokes one or more application resources coupled to the server computing device.

Assignments (1)

References Cited (79)