USER SATISFACTION DETECTION IN A VIRTUAL ASSISTANT
A speech and natural language-based virtual assistant parses user utterances and analyzes them in the context of recent prior actions to detect sentiment and indicators of satisfaction or dissatisfaction. Indicators are stored in a database in association with the prior command and resulting action. Databases can include timestamps, clarifications made by users, and a knowledge graph of facts. Machine learning, applied to the database, train models to deliver improved results in future user engagements.
1 . An arrangement of at least one non-transitory computer readable medium comprising code that, if executed by at least one computer processor comprised by a virtual assistant, would cause the virtual assistant to:
receive a command;
perform an action, responsive to the command, to produce a result for observation by a user;
receive an utterance from the user;
recognize words in the utterance;
analyze the words to produce a satisfaction indicator; and
store the satisfaction indicator in a database to allow the virtual assistant to improve future actions.
2 . The arrangement of claim 1 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to:
responsive to receiving the command, store a timestamp that indicates the approximate time of receiving the command; and
responsive to receiving the utterance, compute a duration of time since receiving the command,
wherein storing the satisfaction indicator is conditional upon the computed duration being less than a specified duration.
3 . The arrangement of claim 1 , wherein analyzing the words to produce a satisfaction indicator comprises searching the words for the presence of one or more negative indicator words.
4 . The arrangement of claim 1 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to, responsive to the satisfaction indicator being negative, perform a second action.
5 . The arrangement of claim 1 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to:
responsive to the satisfaction indicator indicating dissatisfaction, ask the user to provide follow-up information;
receive the follow-up information from the user; and
write the command and the follow-up information to a second computer readable medium.
6 . The arrangement of claim 1 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to:
search the words for the presence of one or more clarification indicator words; and
responsive to identifying that one of the words is a clarification indicator word, recognize associated new information.
7 . The arrangement of claim 6 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to create a fact in a knowledgebase with the new information.
8 . The at least one non-transitory computer readable medium of claim 6 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to replace a fact in a knowledgebase with the new information.
9 . The at least one non-transitory computer readable medium of claim 1 , wherein analyzing the words to produce a satisfaction indicator comprises:
interpreting the words to produce a sentiment; and
producing the satisfaction indicator to indicate dissatisfaction responsive to the sentiment being negative.
10 . An arrangement of at least one non-transitory computer readable medium comprising code that, if executed by at least one computer processor comprised by a virtual assistant, would cause the virtual assistant to:
receive a command;
perform an action, responsive to the command, to produce a result for observation by a user;
receive an utterance from the user;
recognize words in the utterance;
determine if the words include at least one indicator word to determine a satisfaction indicator; and
train a behavioral model by performing a machine learning algorithm on the command, as labelled by the satisfaction indicator.
11 . The arrangement of claim 10 , wherein the code, if executed by the at least one computer processor, would further cause the virtual assistant to:
receive a second command; and
perform a second action according to the behavioral model applied to the second command.
12 . A method of training a virtual assistant from user feedback, the method comprising:
receiving a command from a user by the virtual assistant;
performing an action, responsive to the command, to produce a result for observation by the user;
receiving an utterance from the user;
recognizing words in the utterance;
determining if the words include at least one indicator word to determine a satisfaction indicator; and
configuring a behavioral model, using a machine learning algorithm, to train the virtual assistant to perform desirable actions.
13 . The method of claim 12 further comprising, responsive to the satisfaction indicator being negative, performing a second action.
14 . The method of claim 12 further comprising:
responsive to the satisfaction indicator indicating dissatisfaction, asking the user to provide follow-up information;
receiving the follow-up information from the user; and
writing the command and the follow-up information to a computer readable medium.
15 . The method of claim 12 further comprising:
searching the words for the presence of one or more clarification indicator words; and
responsive to identifying that one of the words is a clarification indicator word, recognizing associated new information.
16 . The method of claim 15 further comprising creating a fact in a knowledgebase with the new information.
17 . The method of claim 12 further comprising:
interpreting the words to produce a sentiment; and
producing the satisfaction indicator to indicate dissatisfaction responsive to the sentiment being negative.