Scalable high-accuracy transactional agents
Aspects of the disclosure are directed to a transactional agent for user interactions. The agent can seamlessly respond to user requests in a conversational manner while maintaining the conversational state. The agent can include a multi-stage modular model architecture, including a semantic understander and a semantic matcher. The semantic understander can be configured to understand common conversation conventions and/or patterns to produce a structure representation of a user request. The semantic matcher can be configured to map items and modifiers to product entries for a particular domain.
1 . A method for processing a conversational transaction comprising:
receiving, by one or more processors, a first conversational input associated with a user request;
generating, by the one or more processors, a first structured representation of the first conversational input using an instruction-prompted machine learning model;
receiving, by the one or more processors, a second conversational input associated with the user request;
generating, by the one or more processors, a second structured representation of the second conversational input using the instruction-prompted machine learning model based on the first structured representation;
matching, by the one or more processors, the second structured representation with an action for responding to the user request using an encoder-based machine learning model, wherein the matching comprises:
generating an embedding to represent the second structured representation; and
matching the embedding with one or more embeddings representing candidate actions for responding to the user request; and
outputting, by the one or more processors, the action for responding to the user request.
2 . The method of claim 1 , wherein:
the first and second conversational inputs are vocal utterances; and
the method further comprises converting, by the one or more processors, at least one of the vocal utterances to a textual utterance.
3 . The method of claim 2 , further comprising:
determining, by the one or more processors, that the textual utterance is a fragmented request; and
waiting, by the one or more processors, for the remainder of the user request in response to determining the textual utterance is a fragmented request.
4 . The method of claim 3 , wherein determining the textual utterance is a fragmented request is based on at least one of a processing via an endpointing model, a last word in the textual utterance, or processing via a large language model (LLM).
5 . The method of claim 1 , wherein generating the second structured representation further comprises updating actions based on a current action state, a conversation history, and the first and second conversational inputs.
6 . The method of claim 1 , wherein outputting the action for responding to the user request further comprises outputting further clarification is needed in response to multiple matching actions.
7 . A system comprising:
one or more processors; and
one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for processing a conversational transaction, the operations comprising:
receiving a first conversational input associated with a user request;
generating a first structured representation of the first conversational input using an instruction-prompted machine learning model;
receiving a second conversational input associated with the user request;
generating a second structured representation of the second conversational input using the instruction-prompted machine learning model based on the first structured representation;
matching the second structured representation with an action for responding to the user request using an encoder-based machine learning model, wherein the matching comprises:
generating an embedding to represent the second structured representation; and
matching the embedding with one or more embeddings representing candidate actions for responding to the user request; and
outputting the action for responding to the user request.
8 . The system of claim 7 , wherein:
the first and second conversational inputs are vocal utterances; and
the operations further comprise converting at least one of the vocal utterances to a textual utterance.
9 . The system of claim 8 , wherein the operations further comprise:
determining that the textual utterance is a fragmented request; and
waiting for the remainder of the user request in response to determining the textual utterance is a fragmented request.
10 . The system of claim 9 , wherein determining the textual utterance is a fragmented request is based on at least one of a processing via an endpointing model, a last word in the textual utterance, or processing via a large language model (LLM).
11 . The system of claim 7 , wherein generating the second structured representation further comprises updating actions based on a current action state, a conversation history, and the first and second conversational inputs.
12 . The system of claim 7 , wherein outputting the action for responding to the user request further comprises outputting further clarification is needed in response to multiple matching actions.
13 . A non-transitory computer readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for processing a conversational transaction, the operations comprising:
receiving a first conversational input associated with a user request;
generating a first structured representation of the first conversational input using an instruction-prompted machine learning model;
receiving a second conversational input associated with the user request;
generating a second structured representation of the second conversational input using the instruction-prompted machine learning model based on the first structured representation;
matching the second structured representation with an action for responding to the user request using an encoder-based machine learning model, wherein the matching comprises:
generating an embedding to represent the second structured representation; and
matching the embedding with one or more embeddings representing candidate actions for responding to the user request; and
outputting the action for responding to the user request.
14 . The non-transitory computer readable medium of claim 13 , wherein:
the first and second conversational inputs are vocal utterances; and
the operations further comprise:
converting at least one of the vocal utterances to a textual utterance;
determining that the textual utterance is a fragmented request; and
waiting for the remainder of the user request in response to determining the textual utterance is a fragmented request.
15 . The non-transitory computer readable medium of claim 13 , wherein generating the second structured representation further comprises updating actions based on a current action state, a conversation history, and the first and second conversational inputs.