Systems and methods for adaptive human-machine interaction and automatic behavioral assessment
Systems and methods for human-machine interaction using a conversation system.
1. A method for controlling a robot comprising:
a conversation system controlling multi-modal output of the robot in accordance with selected conversational content, the multi-modal output controlled by the conversation system including a speaker to output sound, an arm assembly to perform interactive gestures, and a face-like display screen to display images to control facial expressions;
a control system providing first event information to the conversation system based on sensing of a human interaction participant by at least one sensor of the robot during processing of the selected conversational content by the conversation system;
the conversation system controlling the multi-modal output of the robot in accordance with the first event information;
an evaluation system updating conversational content used by the conversation system based on the sensing of the human interaction participant by the at least one sensor of the robot;
the evaluation system providing evaluation results to an external client device based on the sensing of the human interaction participant; and
a goal authoring system providing a user interface to a client device, wherein the user interface includes at least one field for receiving user-input specifying at least a first goal, at least one field for receiving user-input specifying a goal evaluation module of the robot that is to be used to evaluate the first goal, at least one field for receiving user-input specifying at least a first goal level of the first goal, and at least one field for receiving user-input specifying at least a first human interaction participant support level of the first goal level.
2. The method of claim 1 , further comprising: the conversation system performing a first conversation behavior associated with second event information received from the control system, in accordance with the selected conversational content.
3. The method of claim 1 , further comprising: the control system or the evaluation system selecting the selected content based on:
the sensing of the human interaction participant;
the evaluation results;
content associated with successful evaluation results of similar human interaction participants;
history of previously used conversational content;
history of previously used conversational content and related evaluation results;
a specified conversational content schedule; or
a content selection instruction received from an external system.
4. The method of claim 1 , wherein the evaluation system updates the conversational content based on:
the sensing of the human interaction participant;
the evaluation results;
content associated with successful evaluation results of similar human interaction participants;
history of previously used conversational content;
history of previously used conversational content and related evaluation results;
a specified interaction content schedule; or
a content selection instruction received from an external system.
5. The method of claim 1 , wherein the selected conversational content specifies at least a first robot output and at least a first participant input associated with the first robot output, and at least one output of the selected conversational content is a conditional output that is associated with a goal, a goal level, or a human interaction participant support level.
6. The method of claim 5 , wherein a goal is selectively enabled based on: the sensing of the human interaction participant; the evaluation results; content associated with successful evaluation results of similar human interaction participants; history of previously used conversational content; history of previously used conversational content and related evaluation results; a specified conversational content schedule; or an instruction received from an external system.
7. The method of claim 5 , wherein a goal level is selectively enabled based on: the sensing of the human interaction participant; the evaluation results; content associated with successful evaluation results of similar human interaction participants; history of previously used conversational content; history of previously used conversational content and related evaluation results; a specified conversational content schedule; or an instruction received from an external system.
8. The method of claim 5 , wherein a participant support level is selectively enabled based on: the sensing of the human interaction participant; the evaluation results; content associated with successful evaluation results of similar human interaction participants; history of previously used conversational content; history of previously used conversational content and related evaluation results; a specified conversational content schedule; or an instruction received from an external system.
9. The method of claim 1 , wherein the selected conversational content relates to therapy and the evaluation results relate to an assessment of the human interaction participant's responsiveness to the therapy.
10. The method of claim 1 , further comprising:
a content authoring system generating the selected conversational content based on user-input received via a user input device.
11. The method of claim 1 , further comprising: a conversation testing system controlling a multi-modal output system of the robot, responsive to user-input received via a user input device of the conversation testing system, and the conversation testing system receiving event information from the robot indicating a human response sensed by the robot.
12. The method of claim 1 , further comprising: the goal authoring system automatically defining the goal level for the first goal.
13. The method of claim 1 , further comprising: the goal authoring system automatically defining the first human interaction participant support level for the first goal level.
14. The method of claim 1 , further comprising: the goal authoring system determining available goal evaluation modules of the robot by communicating with the robot, and providing to the client device a user interface that identifies the determined available goal evaluation modules of the robot.
15. An apparatus to control a robot comprising:
a conversation system controlling multi-modal output of the robot in accordance with selected conversational content, the multi-modal output controlled by the conversation system including a speaker to output sound, an arm assembly to perform interactive gestures, and a face-like display screen to display images to control facial expressions;
a control system providing first event information to the conversation system based on sensing of a human interaction participant by at least one sensor of the robot during processing of the selected conversational content by the conversation system;
the conversation system controlling multi-modal output of the robot in accordance with the first event information;
an evaluation system providing evaluation results to an external client device based on the sensing of the human interaction participant by the at least one sensor of the robot; and
a goal authoring system providing a user interface to a client device, wherein the user interface includes at least one field for receiving user-input specifying at least a first goal, at least one field for receiving user-input specifying a goal evaluation module of the robot that is to be used to evaluate the first goal, at least one field for receiving user-input specifying at least a first goal level of the first goal, and at least one field for receiving user-input specifying at least a first human interaction participant support level of the first goal level.
16. A method for controlling a machine comprising:
a conversation system controlling multi-modal output of the machine in accordance with selected conversational content, the multi-modal output controlled by the conversation system including a speaker to output sound, an arm assembly to perform interactive gestures, and a face-like display screen to display images to control facial expressions;
a control system providing first event information to the conversation system based on sensing of a human interaction participant by at least one sensor of the machine during processing of the selected conversational content by the conversation system;
the conversation system controlling multi-modal output of the robot in accordance with the first event information;
an evaluation system providing evaluation results to an external client device based on the sensing of the human interaction participant by the at least one sensor of the machine, wherein a participant support level is selected based on the evaluation results; and
a goal authoring system providing a user interface to a client device, wherein the user interface includes at least one field for receiving user-input specifying at least a first goal, at least one field for receiving user-input specifying a goal evaluation module of the robot that is to be used to evaluate the first goal, at least one field for receiving user-input specifying at least a first goal level of the first goal, and at least one field for receiving user-input specifying at least a first human interaction participant support level of the first goal level.