Iteratively updating parameters for dialog states
An interaction assistant conducts multiple turn interaction dialogs with a user in which context is maintained between turns, and the system manages the dialog to achieve an inferred goal for the user. The system includes a linguistic interface to a user and a parser for processing linguistic events from the user. A dialog manager of the system is configured to receive alternative outputs from the parser, and selecting an action and causing the action to be performed based on the received alternative outputs. The system further includes a dialog state for an interaction with the user, and the alternative outputs represent alternative transitions from a current dialog state to a next dialog state. The system further includes a storage for a plurality of templates, and wherein each dialog state is defined in terms of an interrelationship of one or more instances of the templates.
1. A method for determining parameter values for a plurality of components of an interaction system, the system being configured to process ordered pluralities of events, the events including linguistic events including speech utterances, and application related events; the processing of events including parsing of linguistic events via automatic speech recognition to produce text-based representations of the speech utterances, determining an ordered plurality of dialog states, and determining an ordered plurality of output actions from an ordered plurality of events corresponding to the ordered plurality of dialog states; and the method comprising:
collecting a training data set including one or more ordered pluralities of events and, for each ordered plurality of events, a corresponding ordered plurality of output actions;
repeating an iteration, each iteration including processing an ordered plurality of events and a corresponding ordered plurality of output actions by:
processing the ordered plurality of events using current parameter values of the system, the processing including determining an ordered plurality of dialog states from the ordered plurality of events,
determining an ordered plurality of output actions from the ordered plurality of dialog states, and
using a comparison of the determined ordered plurality of output actions and the collected ordered plurality of output actions to update trainable parameter values of the plurality of components of the interaction system; and
completing the repeating of the iterations upon reaching a stopping condition; and
setting the trainable parameter values for the plurality of components of the interaction system using a result of the iterations.
2. The method of claim 1 wherein the components of the system include at least one of an automated speech recognizer, a parser, and a dialog manager for selecting actions proposed by the parser.
3. The method of claim 2 wherein the trainable parameter values of the components include neural network parameters and/or machine learning parameters.
4. The method of claim 1 , wherein collecting the training data set includes:
recognizing an exemplary linguistic event during natural language interaction with a human user;
presenting the exemplary linguistic event to a human trainer; and
receiving, from the human trainer, an exemplary output action to associate with the exemplary linguistic event.
5. The method of claim 1 , wherein updating the trainable parameter values of the plurality of components of the interaction system includes selecting a best weighted execution trace and incrementing the trainable parameter values based at least on a comparison between the best weighted execution trace and the ordered plurality of output actions.
6. The method of claim 1 , wherein updating the trainable parameters of the interaction system includes:
assessing a comparison score based on comparing the determined ordered plurality of output actions to an exemplary ordered plurality of output actions from the training data set; and
updating the trainable parameters of the interaction system based on the assessed comparison score to improve a likelihood of generating a correct ordered plurality of output actions with regard to a future ordered plurality of events.
7. A method for training a plurality of components of an interaction system, the method comprising:
collecting a training data set including one or more ordered pluralities of events, and, for each ordered plurality of events, a corresponding ordered plurality of output actions, wherein collecting the training data set includes:
recognizing an exemplary linguistic event during natural language interaction with a human user;
presenting the exemplary linguistic event to a human trainer; and
receiving, from the human trainer, an exemplary output action to associate with the exemplary linguistic event; and
for an exemplary ordered plurality of events and a corresponding exemplary ordered plurality of output actions:
generating an ordered plurality of output actions based on the exemplary ordered plurality of events and current parameter values for trainable parameters of the interaction system;
comparing the generated ordered plurality of output actions to the exemplary ordered plurality of output actions; and
updating the trainable parameters of the interaction system based on the comparison.
8. The method of claim 7 , wherein receiving the exemplary output action from the human trainer includes presenting the human trainer with a plurality of candidate output action plans, and receiving input from the human trainer indicating a most appropriate candidate output action plan.
9. The method of claim 7 , wherein the ordered plurality of events includes a speech utterance, and the method further comprises performing automatic speech recognition to produce a text-based representation of the speech utterance.
10. The method of claim 7 , wherein the components of the system include at least one of an automated speech recognizer, a parser, and a dialog manager for selecting actions proposed by the parser.
11. The method of claim 10 , wherein the trainable parameter values of the components include neural network parameters and/or machine learning parameters.
12. The method of claim 7 , wherein updating the trainable parameter values of the plurality of components of the interaction system includes selecting a best weighted execution trace and incrementing the trainable parameter values based at least on a comparison between the best weighted execution trace and the ordered plurality of output actions.
13. The method of claim 7 , wherein updating the trainable parameters of the interaction system includes:
assessing a comparison score based on comparing the determined ordered plurality of output actions to an exemplary ordered plurality of output actions from the training data set; and
updating the trainable parameters of the interaction system based on the assessed comparison score to improve a likelihood of generating a correct ordered plurality of output actions with regard to a future ordered plurality of events.
14. A method for training a plurality of components of an interaction system, the method comprising:
collecting a training data set including one or more ordered pluralities of events, and, for each ordered plurality of events, a corresponding ordered plurality of output actions;
for an exemplary ordered plurality of events and a corresponding exemplary ordered plurality of output actions:
generating an ordered plurality of output actions based on the exemplary ordered plurality of events and current parameter values for trainable parameters of the interaction system;
assessing a comparison score based on comparing the generated ordered plurality of output actions to the exemplary ordered plurality of output actions; and
updating the trainable parameters of the interaction system based on the assessed comparison score to improve a likelihood of generating a correct ordered plurality of output actions with regard to a future ordered plurality of events.
15. The method of claim 14 , wherein the ordered plurality of events includes a speech utterance, and the method further comprises performing automatic speech recognition to produce a text-based representation of the speech utterance.
16. The method of claim 14 , wherein the components of the system include at least one of an automated speech recognizer, a parser, and a dialog manager for selecting actions proposed by the parser.
17. The method of claim 16 , wherein the trainable parameter values of the components include neural network parameters and/or machine learning parameters.
18. The method of claim 14 , wherein updating the trainable parameter values of the plurality of components of the interaction system includes selecting a best weighted execution trace and incrementing the trainable parameter values based at least on a comparison between the best weighted execution trace and the ordered plurality of output actions.
19. The method of claim 14 , wherein collecting the training data set includes:
recognizing an exemplary linguistic event during natural language interaction with a human user;
presenting the exemplary linguistic event to a human trainer; and
receiving, from the human trainer, an exemplary output action to associate with the exemplary linguistic event.
20. The method of claim 19 , wherein receiving the exemplary output action from the human trainer includes presenting the human trainer with a plurality of candidate output action plans, and receiving input from the human trainer indicating a most appropriate candidate output action plan.