Intent-based query and response routing between users and backend services
For a seamless and robust artificial intelligence-based assistant experience, an intent-based query and response router has been designed to operate as an intelligent layer between a user and multiple backend services that may respond to one or more queries over the course of a conversation with the user. The query router interacts with an intent classification service to obtain an intent classification for a prompt that is based on a user query. The query router uses the intent classification, which is used as an identifier of a backend service, to route the user query to an appropriate one (or more) of the backend services. When a response is detected, the query router determines a corresponding conversation and provides the response for the conversation.
1. A method comprising:
submitting a first prompt to a set of one or more large language model (LLM) based intent classifiers to obtain at least a first intent classification, wherein the first prompt is based on a first user query received via a first interface;
routing the first user query to at least one of a plurality of backend services based, at least partly, on the first intent classification;
detecting a first query response from at least one of the plurality of backend services;
determining that the first query response corresponds to the first user query; and
responding to the first user query based, at least in part, on the first query response.
2. The method of claim 1 , wherein the set of one or more LLM based intent classifiers is trained with training data comprising tuples of example prompts and backend service identifiers.
3. The method of claim 1 , wherein routing the first user query to at least one of the plurality of backend services comprises:
creating a message with the first user query and with the first intent classification; and
publishing the message to a first message queue accessed by the plurality of backend services.
4. The method of claim 3 further comprising:
obtaining a second intent classification from the set of LLM based intent classifiers based on the first prompt or a second prompt generated based on the first user query; and
creating the message with the second intent classification in addition to the first intent classification or creating a second message with the first prompt and the second intent classification and publishing the second message to the first message queue.
5. The method of claim 1 further comprising:
creating the first prompt based on the first user query and at least one of conversation history based context and customer based context data.
6. The method of claim 5 , wherein creating the first prompt is responsive to determining that a response from the set of one or more LLM based intent classifiers to a second prompt that was submitted prior to the first prompt and that is also based on the first user query does not indicate a defined backend service identifier.
7. The method of claim 1 further comprising pre-processing the first user query to be a valid prompt.
8. The method of claim 1 further comprising:
detecting multiple query responses from the plurality of backend services, wherein the multiple query responses include the first query response;
determining that a query state for the first user query is complete; and
aggregating the multiple query responses into an aggregate response,
wherein responding to the first user query comprises responding with the aggregate response.
9. The method of claim 8 , wherein aggregating the multiple query responses into the aggregate response comprises one of collating the multiple query responses, summarizing the multiple query responses, and merging the multiple query responses.
10. The method of claim 8 further comprising ranking the multiple query responses and filtering the multiple query responses based, at least, in part on the ranking.
11. The method of claim 1 further comprising editing the first query response based, at least in part, on an attribute of a digital identity associated with the first user query.
12. A non-transitory, machine-readable medium having program code stored thereon, the program code comprising instructions to:
submit a first prompt to a set of one or more large language model (LLM) based intent classifiers to obtain at least a first intent classification, wherein the first prompt is based on a first user query received via a first interface;
communicate the first user query to at least one of a plurality of backend services based, at least partly, on the first intent classification;
detect a first query response from at least one of the plurality of backend services;
determine that the first query response corresponds to the first user query; and
respond to the first user query based, at least in part, on the first query response.
13. The machine-readable medium of claim 12 , wherein the first interface is an application programming interface.
14. The machine-readable medium of claim 12 , wherein the instructions to communicate the first user query comprise instructions to:
create a message with the first user query and with the first intent classification; and
publish the message to a first message queue accessed by the plurality of backend services.
15. The machine-readable medium of claim 14 , wherein the program code further has stored thereon instructions to:
obtain a second intent classification from the set of LLM based intent classifiers based on the first prompt or a second prompt generated based on the first user query; and
create the message with the second intent classification in addition to the first intent classification or create a second message with the first prompt and the second intent classification and publish the second message to the first message queue.
16. The machine-readable medium of claim 12 , wherein the program code to communicate the first user query comprises instructions to:
for each intent classification from the set of LLM based intent classifiers for the first user query, supply the first user query to the one of the plurality of backend services identified by the intent classification.
17. The machine-readable medium of claim 12 , wherein the program code further comprises instructions to:
detect multiple query responses from the plurality of backend services including the first query response;
determine that a query state for the first user query is complete; and
aggregate the multiple query responses into an aggregate response,
wherein the instructions to respond to the first user query comprise instructions to respond with the aggregate response.
18. An apparatus comprising:
a processor; and
a machine-readable medium having instructions stored thereon that are executable by the processor to cause the apparatus to,
submit a first prompt to a set of one or more large language model (LLM) based intent classifiers to obtain at least a first intent classification, wherein the first prompt is based on a first user query received via a first interface;
communicate the first user query to at least one of a plurality of backend services based, at least partly, on the first intent classification;
detect a first query response from at least one of the plurality of backend services;
determine that the first query response corresponds to the first user query; and
respond to the first user query based, at least in part, on the first query response.
19. The apparatus of claim 18 , wherein the instructions to communicate the first user query comprise instructions executable by the processor to cause the apparatus to:
for each intent classification from the set of LLM based intent classifiers for the first user query, supply the first user query to the one of the plurality of backend services identified by the intent classification.
20. The apparatus of claim 18 , wherein the machine-readable medium further has stored thereon instructions executable by the processor to cause the apparatus to:
detect multiple query responses from the plurality of backend services including the first query response;
determine that a query state for the first user query is complete; and
aggregate the multiple query responses into an aggregate response,
wherein the instructions to respond to the first user query comprise instructions to respond with the aggregate response.