System and method of providing conversational visual prosody for talking heads
View Patent ↗A system and method of controlling the movement of a virtual agent while the agent is listening to a human user during a conversation is disclosed. The method comprises receiving speech data from the user, performing a prosodic analysis of the speech data and controlling the virtual agent movement according to the prosodic analysis.
1. A method of providing user-specific animated entity movements during a conversation with a user, the method comprising:
determining a parameter associated with a culture of a user;
based on the parameter, selecting data associated with listening movements of an animated entity based on the user's culture; and
controlling listening movement of the animated entity when the animated entity is listening to the user based on the selected data and a prosodic analysis of user's speech such that the listening movement of the animated entity is consistent with the user's culture.
2. The method of providing user-specific animated entity movements during a conversation with the user of claim 1 , wherein the parameter associated with the user is further associated with a language spoken by the user.
3. The method of providing user-specific animated entity movements during a conversation with the user of claim 1 , wherein the parameter associated with the user is further associated with an age of the user.
4. The computer-readable medium of claim 1 , wherein the parameter associated with the user is further associated with a dialect spoken by the user.
5. The computer-readable medium of claim 1 , wherein the parameter associated with the user is further associated with an ethnicity of the user.
6. A computing device for providing user-specific animated entity movements during a conversation with a user, the computing device comprising:
a module configured to determine a parameter associated with a culture of a user;
a module configured to, based on the parameter, select data associated with listening movements of an animated entity based on the user's culture; and
a module configured to control listening movement of the animated entity when the animated entity is listening to the user based on the selected data and a prosodic analysis of user's speech such that the listening movement of the animated entity is consistent with the user's culture.
7. The computing device of claim 6 , wherein the parameter associated with the user is further associated with a language spoken by the user.
8. The computing device of claim 6 , wherein the parameter associated with the user is further associated with an age of the user.
9. The computing device of claim 6 , wherein the parameter associated with the user is further associated with a dialect spoken by the user.
10. The computing device of claim 6 , wherein the parameter associated with the user is further associated with an ethnicity of the user.
11. A computer-readable medium storing instructions for controlling a computing device to provide user-specific animated entity movements during a conversation with a user, the instructions comprising:
determining a parameter associated with a culture of a user;
based on the parameter, selecting data associated with listening movements of an animated entity based on the user's culture; and
controlling listening movement of the animated entity when the animated entity is listening to the user based on the selected data and a prosodic analysis of user's speech such that the listening movement of the animated entity is consistent with the user's culture.
12. The computer-readable medium of claim 11 , wherein the parameter associated with the user is further associated with a language spoken by the user.
13. The computer-readable medium of claim 11 , wherein the parameter associated with the user is further associated with an age of the user.
14. A method of controlling virtual agent listening movement, the method comprising:
receiving speech data from the user;
performing a prosodic analysis of the speech data;
based on the prosodic analysis, selecting matching prosody patterns from a listening database;
determining a culture associated with a user; and
controlling virtual agent listening movement according to the prosodic analysis and further based on data associated with the determined culture of the user.
15. The method of claim 14 , wherein the prosodic analysis further comprises using a model to determine what virtual agent movement to associate with the speech data.