Modeling characters that interact with users as part of a character-as-a-service implementation
In various embodiments, a character engine models a character that interacts with users The modeling techniques include evaluating user input data that is associated with a user device to identify a user intent and an assessment domain, selecting a first set of inference algorithms from a plurality of inference algorithms based, at least in part, on the user intent and the assessment domain, and applying the user intent and the assessment domain to the first set of inference algorithms to generate a plurality of inferences. The modeling techniques further include consolidating the plurality of inferences into a consolidated inference, applying a second set of inference algorithms from the plurality of inference algorithms to the consolidated inference to generate a context-specific inference, computing the character response to the user input data based on the user input data, the context-specific inference, and data representing knowledge associated with a character, and causing the user device to output the character response to the user.
1 . A computer-implemented method for generating a character response during an interaction with a user, the method comprising:
generating, by a processor, recognition data from user input data that is generated by one or more sensors associated with a user device of a user;
evaluating, by the processor, the recognition data to identify a user intent of the user and an assessment domain associated with the user intent;
selecting, from a plurality of inference algorithms based, at least in part, on the user intent and the assessment domain, a first set of at least two inference algorithms;
applying the user intent and the assessment domain to each inference algorithm in the first set of inference algorithms to generate a plurality of inferences;
consolidating the plurality of inferences into a consolidated inference by weighing the plurality of inferences based on a plurality of confidence factors;
applying a second set of inference algorithms from the plurality of inference algorithms to the consolidated inference to generate a context-specific inference, wherein the first set of inference algorithms and the second set of inference algorithms are mutually exclusive;
computing, based on the user input data, the context-specific inference, and data representing knowledge associated with a character, a content of the character response to the user input data;
determining an expression of the character response based on one or more capabilities of the user device;
configuring, by the processor, the user device to output the content of the character response and the expression of the user device to the user;
generating training data based on at least one of the recognition data, the user intent, the context-specific inference, the character response, or one or more additional interactions between the user and the character;
identifying, for an item of data included in the training data, one or more inference algorithms included in the plurality of inference algorithms and associated with (i) a data type included in the item of data or (ii) a type of inference operation performed on the item of data;
training, in an offline training mode and using at least the item of data, at least one of the one or more identified inference algorithms included in the plurality of inference algorithms to generate at least one updated inference algorithm;
generating, by the at least one updated inference algorithm, an additional content of an additional character response to additional user input data associated with the user, wherein the additional character response is generated based on at least one updated inference outputted by the at least one updated inference algorithm and at least one updated confidence factor associated with the at least one updated inference;
determining an additional expression of the additional character response based on one or more additional capabilities of an additional user device, wherein the additional expression of the additional character response differs from the expression of the character response; and
configuring the additional user device to output the additional content of the additional character response and the additional expression of the additional character response.
2 . The computer-implemented method of claim 1 , further comprising:
for each inference in the plurality of inferences, generating a corresponding merit rating,
wherein consolidating the plurality of inferences comprises selecting an inference corresponding to a highest merit rating in a plurality of merit ratings corresponding to the plurality of inferences.
3 . The computer-implemented method of claim 1 , wherein the plurality of inference algorithms includes at least one of: a theory-of-mind (TOM) system, a Markov model, a neural network, a computer vision system, or a support vector machine (SVM).
4 . The computer-implemented method of claim 1 , wherein the plurality of inferences includes a predicted user intent based on the user input data.
5 . The computer-implemented method of claim 1 , wherein the user device comprises a robot, a walk around character, or a toy.
6 . The computer-implemented method of claim 1 , wherein the data representing knowledge associated with the character includes information obtained from at least one of a World Wide Web, a script, a book, or a user-specific history.
7 . The computer-implemented method of claim 1 , wherein the expression of the character response and the additional expression of the additional character response comprise at least one of: text, speech, a gesture, a facial expression, a movement, a physical action, a sound, or an image.
8 . The computer-implemented method of claim 1 , wherein:
computing the content of the character response comprises selecting, based at least on the context-specific inference and from a plurality of sets of personality characteristics, a first set of personality characteristics for generating the content of the character response; and
the first set of personality characteristics comprises a first plurality of parameters, each parameter being associated with a personality dimension.
9 . The computer-implemented method of claim 8 , wherein:
determining the expression of the character response comprises selecting, based at least on the context-specific inference and from the plurality of sets of personality characteristics, a second set of personality characteristics for generating the expression of the character response, and
the second set of personality characteristics comprises a second plurality of parameters that is different from the first plurality of parameters.
10 . The computer-implemented method of claim 1 , wherein the assessment domain is further generated from a mode of interaction with the user associated with the user input data.
11 . The computer-implemented method of claim 1 , further comprising:
combining a set of historical data associated with the user with at least one of the consolidated inference, the context-specific inference, or the character response; and
generating a set of user-specific consolidated data.
12 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to generate a character response during an interaction with a user by performing the steps of:
generating recognition data from user input data that is generated by one or more sensors associated with a user device of a user;
evaluating the recognition data to identify a user intent of the user and an assessment domain associated with the user intent;
selecting, from a plurality of inference algorithms based, at least in part, on the user intent and the assessment domain, a first set of at least two inference algorithms;
applying the user intent and the assessment domain to each inference algorithm in the first set of inference algorithms to generate a plurality of inferences;
consolidating the plurality of inferences into a consolidated inference by weighing the plurality of inferences based on a plurality of confidence factors;
applying a second set of inference algorithms from the plurality of inference algorithms to the consolidated inference to generate a context-specific inference, wherein the first set of inference algorithms and the second set of inference algorithms are mutually exclusive;
computing, based on the user input data, the context-specific inference, and data representing knowledge associated with a character, the character response to the user input data;
receiving, based on a plurality of interactions with a plurality of users, structured data that includes (i) a set of named entities and (ii) a set of relationships associated with the set of named entities;
generating training data that includes the structured data, the context-specific inference, the character response, and one or more additional interactions between the user and the character;
identifying, for an item of data included in the training data, one or more inference algorithms included in the plurality of inference algorithms and associated with (i) a data type included in the item of data or (ii) a type of inference operation performed on the item of data;
training, in an offline training mode and using at least the item of data, at least one of the one or more identified inference algorithms included in the plurality of inference algorithms to generate at least one updated inference algorithm; and
generating, by the at least one updated inference algorithm, an additional character response to additional user input data associated with the user, wherein the additional character response is generated based on at least one updated inference outputted by the at least one updated inference algorithm and at least one updated confidence factor associated with the at least one updated inference.
13 . The one or more non-transitory computer-readable media of claim 12 , further storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the step of:
determining an expression of the character response based on one or more capabilities of a robot corresponding to the user device; and
configuring the robot to output the expression of the character response to the user,
wherein the expression of the character response comprises at least one of a text output, a voice output, a movement, a gesture, or a facial expression.
14 . The one or more non-transitory computer-readable media of claim 12 , wherein the plurality of inference algorithms includes at least one of: a theory-of-mind (TOM) system, a Markov model, a neural network, a computer vision system, or a support vector machine (SVM).
15 . The one or more non-transitory computer-readable media of claim 12 , wherein the plurality of inferences includes a predicted user intent based on the user input data.
16 . The one or more non-transitory computer-readable media of claim 12 , wherein the data representing knowledge associated with the character includes information obtained from at least one of a World Wide Web, a script, a book, or a user-specific history.
17 . The one or more non-transitory computer-readable media of claim 12 , wherein:
computing the character response comprises:
selecting, based at least on the context-specific inference and from a plurality of sets of personality characteristics, a first set of personality characteristics for generating content of the character response, and
selecting, based at least on the context-specific inference and from the plurality of sets of personality characteristics, a second set of personality characteristics for generating an expression of the content of the character response,
the first set of personality characteristics comprises a first plurality of parameters, each parameter being associated with a personality dimension; and
the second set of personality characteristics comprises a second plurality of parameters that is different from the first plurality of parameters.
18 . The one or more non-transitory computer-readable media of claim 12 , wherein the assessment domain is further generated from a mode of interaction with the user associated with the user input data.
19 . The one or more non-transitory computer-readable media of claim 12 , further storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of:
combining a set of historical data associated with the user with at least one of the consolidated inference, the context-specific inference, or the character response; and
generating a set of user-specific consolidated data.
20 . A computer-implemented method for generating a character response during an interaction with a user, the method comprising:
generating, by a processor, recognition data from user input data that is generated by one or more sensors associated with a user device of a user;
evaluating, by the processor, the recognition data to identify a user intent of the user and an assessment domain associated with the user intent;
selecting at least two of a plurality of inference algorithms based, at least in part, on the user intent and the assessment domain;
applying the user intent and the assessment domain to each of the two or more selected inference algorithms to generate a plurality of inferences;
consolidating the plurality of inferences into a consolidated inference by weighing the plurality of inferences based on a plurality of confidence factors;
determining one or more expression capabilities of the user device, wherein the one or more expression capabilities comprise at least one of a graphical processing power, a facial expression, a speech output, a movement, or a gesture;
applying, based on the one or more expression capabilities of the user device, at least one of the plurality of inference algorithms to the consolidated inference to generate a context-specific inference;
computing, based on the user input data, the context-specific inference, and data representing knowledge associated with a character, the character response to the user input data;
configuring the user device to output the character response to the user based on the one or more expression capabilities of the user device;
receiving, based on a plurality of interactions with a plurality of users, structured data that includes (i) a set of named entities and (ii) a set of relationships associated with the set of named entities;
training, in an offline training mode, at least one inference algorithm included in the plurality of inference algorithms based on training data that includes the structured data, the user input data, the character response, and one or more additional interactions between the user and the character to generate at least one updated inference algorithm, wherein the at least one inference algorithm is identified for training on a data item included in the training data, based at least on a type of inference operation performed by the at least one inference algorithm;
generating, by the at least one updated inference algorithm, an additional character response to additional user input data associated with the user, wherein the additional character response is generated based on at least one updated inference outputted by the at least one updated inference algorithm and at least one updated confidence factor associated with the at least one updated inference; and
configuring an additional user device to output the additional character response based on one or more additional expression capabilities of the additional user device, wherein an expression of the additional character response by the additional user device differs from an expression of the character response by the user device.