Interactive character system
A system and method for controlling a synthetic character using a control system displays the character engaged in an activity, receiving a first input from a user, determines whether the input is relevant to the activity, if the input is relevant to the activity, and shows the character react to the input, the character being highly expressive and highly reactive. A system and method for displaying a synthetic character provides speech data, creates modified speech data by modifying at least one of the pitch or duration of at least a portion of the speech data and generates modified speech sounds associated with the character using the modified speech data.
1. A method for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data;
generating speech sounds associated with the character using the modified speech data; and
associating a representation of an emotional state with the character; and
wherein creating modified speech data comprises modifying the speech data based on the emotional state representation and
wherein the emotional state representation is determined at least in part by interaction of the character with others in a shared environment.
2. A method for controlling the voice of a synthetic character that is autonomous and interacts with others in a shared environment, such character having an associated mental state that changes over time at least in part as a function of said mental state, said mental state comprising a dynamic perceptual state that depends on said shared environment, said shared environment including the entities with which the character is interacting, and an action state defining actions to be taken by the character, the method comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data, such modification based at least in part on at least a portion of said mental state; and
generating speech sounds associated with the character using the modified speech data;
wherein the mental state is determined at least in part by interaction of the character with others in a shared environment.
3. The method of claim 2 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
4. The method of claim 2 wherein said interactive synthetic character is displayed within an interactive entertainment product.
5. The method of claim 2 wherein said mental state further comprises a defined emotional state.
6. The method of claim 5 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
7. The method of claim 5 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
8. The method of claim 2 wherein said mental state further comprises a defined motivational or goal state.
9. The method of claim 8 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
10. The method of claim 8 wherein said mental state further comprises a defined emotional state.
11. The method of claim 10 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
12. The method of claim 11 further comprising changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
13. The method of claim 8 wherein said motivation or goal state comprises a plurality of goals and subgoals or motivations and submotivations ordered at least in part in a hierarchical relationship.
14. The method of claim 13 wherein said mental state further comprises a defined emotional state.
15. The method of claim 14 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
16. The method of claim 15 further comprising changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
17. The method of claim 2 , wherein said speech sounds have a natural quality such as might be displayed by a living animal or person.
18. The method of claim 17 , wherein said speech sounds are generated in coordination with movement of said synthetic character so that said coordination has a natural quality such as might be displayed by a living animal, person, or imaginary living creature.
19. The method of claim 18 wherein said interactive synthetic character is displayed within an interactive entertainment product.
20. The method of claim 18 wherein said mental state further comprises a defined emotional state.
21. The method of claim 20 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
22. The method of claim 20 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
23. The method of claim 18 wherein said mental state further comprises a defined motivational or goal state.
24. The method of claim 23 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
25. The method of claim 23 wherein said mental state further comprises a defined emotional state.
26. The method of claim 25 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
27. The method of claim 26 further comprising changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
28. The method of claim 23 wherein said motivation or goal state comprises a plurality of goals and subgoals or motivations and submotivations ordered at least in part in a hierarchical relationship.
29. The method of claim 28 wherein said mental state further comprises a defined emotional state.
30. The method of claim 29 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
31. The method of claim 30 further comprising changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
32. The method of claim 18 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
33. A method for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data; wherein the character has a specified personality; and
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.
34. A method for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data; wherein the character is highly expressive, highly reactive and has a specified personality
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.
35. A method for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data, wherein the character is highly reactive and has at least one of the following group of characteristics: highly expressive, appearing to be intelligent, exhibiting common sense, exhibiting social knowledge, exhibiting knowledge of social norms, having a specified personality
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.
36. A computer readable medium encoding computer readable instructions for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, that cause a processor to perform steps comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
associating a representation of an emotional state with the character; and
wherein the instructions for creating modified speech data comprise instructions for modifying the speech data based on the emotional state representation and
wherein the emotional state representation is determined at least in part by interaction of the character with others in a shared environment.
37. A computer readable medium encoding computer readable instructions for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, that cause a processor to perform steps comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data, wherein the character is highly expressive
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.
38. A computer readable medium encoding computer readable instructions for controlling a synthetic character that is autonomous and interacts with others in a shared environment, such character having an associated mental state that changes over time at least in part as a function of said mental state, said mental state comprising a dynamic perceptual state that depends on said shared environment, said shared environment including the entities with which the character is interacting, and an action state defining actions to be taken by the character, that cause a processor to perform steps comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data, such modification based at least in part on at least a portion of said mental state; and
generating speech sounds associated with the character using the modified speech data
wherein the mental state is determined at least in part by interaction of the character with others in a shared environment.
39. The computer readable medium of claim 38 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
40. The computer readable medium of claim 38 wherein said interactive synthetic character is displayed within an interactive entertainment product.
41. The computer readable medium of claim 38 wherein said mental state further comprises a defined emotional state.
42. The computer readable medium of claim 41 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
43. The computer readable medium of claim 41 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
44. The computer readable medium of claim 38 wherein said mental state further comprises a defined motivational or goal state.
45. The computer readable medium of claim 44 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
46. The computer readable medium of claim 44 wherein said mental state further comprises a defined emotional state.
47. The computer readable medium of claim 46 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
48. The computer readable medium of claim 47 further comprising instructions that cause a processor to perform the step of changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
49. The computer readable medium of claim 44 wherein said motivation or goal state comprises a plurality of goals and subgoals or motivations and submotivations ordered at least in part in a hierarchical relationship.
50. The computer readable medium of claim 49 wherein said mental state further comprises a defined emotional state.
51. The computer readable medium of claim 50 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
52. The computer readable medium of claim 51 further comprising instructions that cause a processor to perform the step of changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
53. The computer readable medium of claim 38 , wherein said speech sounds have a natural quality such as might be displayed by a living animal or person.
54. The computer readable medium of claim 53 , wherein generating said speech sounds comprises generating said speech sounds in coordination with movement of said synthetic character so that said coordination has a natural quality such as might be displayed by a living animal, person, or imaginary living creature.
55. The computer readable medium of claim 54 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
56. The computer readable medium of claim 54 wherein said interactive synthetic character is displayed within an interactive entertainment product.
57. The computer readable medium of claim 54 wherein said mental state further comprises a defined emotional state.
58. The computer readable medium of claim 57 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
59. The computer readable medium of claim 57 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
60. The computer readable medium of claim 54 wherein said mental state further comprises a defined motivational or goal state.
61. The computer readable medium of claim 60 wherein said modification of pitch or duration is performed at least in part using the Synchronous Overlap and Add class of techniques.
62. The computer readable medium of claim 60 wherein said mental state further comprises a defined emotional state.
63. The computer readable medium of claim 62 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
64. The computer readable medium of claim 63 further comprising instructions that cause a processor to perform the step of changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
65. The computer readable medium of claim 60 wherein said motivation or goal state comprises a plurality of goals and subgoals or motivations and submotivations ordered at least in part in a hierarchical relationship.
66. The computer readable medium of claim 65 wherein said mental state further comprises a defined emotional state.
67. The computer readable medium of claim 66 wherein said modification of pitch or duration is based at least in part on the degree of intensity of said defined emotional state.
68. The computer readable medium of claim 67 further comprising instructions for changing said emotional state in response to a failure or success of a motivation or goal present in said motivational or goal state.
69. A computer readable medium encoding computer readable instructions for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, that cause a processor to perform steps comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data;
wherein the character has a specified personality; and
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.
70. A computer readable medium encoding computer readable instructions for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, that cause a processor to perform steps comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data;
wherein the character is highly expressive, highly reactive and has a specified personality; and
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.
71. A computer readable medium encoding computer readable instructions for controlling the voice of a synthetic character, which is autonomous and interacts with others in a shared environment, that cause a processor to perform steps comprising:
providing speech data corresponding to at least a part of an intended communication generated by the character;
creating modified speech data by modifying, by an automatically determined amount, at least one of the pitch or duration of at least a portion of the speech data; and
generating speech sounds associated with the character using the modified speech data, wherein the character is highly reactive and has at least one of the following group of characteristics: highly expressive, appearing to be intelligent, exhibiting common sense, exhibiting social knowledge, exhibiting knowledge of social norms, having a specified personality;
wherein the automatically determined amount is determined, at least in part, by a state of the character that depends, at least in part, on interaction of the character with others in the shared environment.