Multi-dimensional puppet with photorealistic movement

1. A computer system, comprising:

a computation device;

a memory configured to store program instructions, wherein, when executed by the computation device, the program instructions cause the computer system to perform one or more operations comprising:

providing, based at least in part on predetermined parameters, configuration information, and a group of behavioral agents, a dynamic virtual representation comprising a multi-dimensional puppet having one or more attributes of an individual, wherein the dynamic virtual representation is configured to automatically mimic one or more attributes of the individual in a context,

wherein the providing of the dynamic virtual representation comprising the multi-dimensional puppet involves rendering of the multi-dimensional puppet,

wherein the multi-dimensional puppet comprises stereopsis information, and has a photorealistic movement corresponding to movement behaviors of the individual, and

wherein the multi-dimensional puppet comprises: a 3D rig having a shape corresponding to at least a shape of a head and neck of the individual; a neutral layer corresponding to a look and color of at least the face and the neck of the individual; a core region overlay layer with 2D bitmaps for portions of the face and the neck of the individual; and a specular overlay layer that reproduces specular highlights of the individual;

receiving an input corresponding to a user mood and corresponding to user spatial manipulation of or interaction with the multi-dimensional puppet, wherein the input does not explicitly indicate the user mood;

determining, using at least a behavioral agent in the group of behavioral agents, the user mood based at least in part on the input; and

providing, based at least in part on the predetermined parameters, the configuration information, the group of behavioral agents, the determined user mood, and the input, the dynamic virtual representation comprising a revised multi-dimensional puppet having the one or more attributes.

2. The computer system of claim 1 , wherein the group of behavioral agents are configured in a multi-layer hierarchy based at least in part on the configuration information;

wherein a given behavioral agent receives one or more inputs and provides an output corresponding to one or more features associated with an individual; and

wherein the inputs to at least some of the behavioral agents comprise outputs from one or more of the other behavioral agents.

3. The computer system of claim of claim 1 , wherein the providing of the dynamic virtual representation comprising the revised multi-dimensional puppet involves rendering of the revised multi-dimensional puppet; and

wherein the revised multi-dimensional puppet comprises prehension corresponding to the input.

4. The computer system of claim of claim 1 , wherein the photorealistic movement is indistinguishable, to a human, from the movement behaviors or mannerisms of the individual.

5. The computer system of claim of claim 1 , wherein the multi-dimensional puppet comprises photorealistic facial and mouth movement while saying phrases.

6. The computer system of claim of claim 1 , wherein the input corresponds to one of: haptic interaction, a gaze direction, a verbal command, or a gesture.

7. The computer system of claim of claim 1 , wherein the rendering is based at least in part on one or more of: characterization of movement a neck, a mouth and eyes of the individual, phonemes associated with speech of the individual, or deformations of a face of the individual while performing the movement behaviors.

8. The computer system of claim of claim 1 , wherein the multi-dimensional puppet comprises 2D data transposed onto a 3D mesh.

9. The computer system of claim of claim 1 , wherein the portions of the face and the neck comprise: a mouth, eyes, a submandibular triangle, a submental triangle, and a muscular triangle.

10. The computer system of claim of claim 1 , wherein the multi-dimensional puppet comprises looped persistent movements corresponding to the movement behaviors.

11. A non-transitory computer-readable storage medium for use in conjunction with a computer system, the computer-readable storage medium configured to store program instructions that, when executed by the computer system, causes the computer system to perform one or more operations comprising:

providing, based at least in part on predetermined parameters, configuration information, and a group of behavioral agents, a dynamic virtual representation comprising a multi-dimensional puppet having one or more attributes of an individual, wherein the dynamic virtual representation is configured to automatically mimic one or more attributes of the individual in a context,

wherein the providing of the dynamic virtual representation comprising the multi-dimensional puppet involves rendering of the multi-dimensional puppet,

wherein the multi-dimensional puppet comprises stereopsis information, and has a photorealistic movement corresponding to movement behaviors of the individual, and

wherein the multi-dimensional puppet comprises: a 3D rig having a shape corresponding to at least a shape of a head and neck of the individual; a neutral layer corresponding to a look and color of at least the face and the neck of the individual; a core region overlay layer with 2D bitmaps for portions of the face and the neck of the individual; and a specular overlay layer that reproduces specular highlights of the individual;

receiving an input corresponding to a user mood and corresponding to user spatial manipulation of or interaction with the multi-dimensional puppet, wherein the input does not explicitly indicate the user mood;

determining, using at least a behavioral agent in the group of behavioral agents, the user mood based at least in part on the input; and

providing, based at least in part on the predetermined parameters, the configuration information, the group of behavioral agents, the determined user mood, and the input, the dynamic virtual representation comprising a revised multi-dimensional puppet having the one or more attributes.

12. The computer-readable storage medium of claim of claim 11 , wherein the providing of the dynamic virtual representation comprising the revised multi-dimensional puppet involves rendering of the revised multi-dimensional puppet; and

wherein the revised multi-dimensional puppet comprises prehension corresponding to the input.

13. The computer-readable storage medium of claim of claim 11 , wherein the photorealistic movement is indistinguishable, to a human, from the movement behaviors or mannerisms of the individual.

14. The computer-readable storage medium of claim of claim 11 , wherein the multi-dimensional puppet comprises photorealistic facial and mouth movement while saying phrases.

15. The computer-readable storage medium of claim of claim 11 , wherein the input corresponds to one of: haptic interaction, a gaze direction, a verbal command, or a gesture.

16. The computer-readable storage medium of claim of claim 11 , wherein the rendering is based at least in part on one or more of: characterization of movement a neck, a mouth and eyes of the individual, phonemes associated with speech of the individual, or deformations of a face of the individual while performing the movement behaviors.

17. The computer-readable storage medium of claim of claim 11 , wherein the multi-dimensional puppet comprises 2D data transposed onto a 3D mesh.

18. A method for providing a dynamic virtual representation, wherein the method comprises:

by a computer system:

providing, based at least in part on predetermined parameters, configuration information, and a group of behavioral agents, the dynamic virtual representation comprising a multi-dimensional puppet having one or more attributes of an individual, wherein the dynamic virtual representation is configured to automatically mimic one or more attributes of the individual in a context,

wherein the providing of the dynamic virtual representation comprising the multi-dimensional puppet involves rendering of the multi-dimensional puppet,

wherein the multi-dimensional puppet comprises stereopsis information, and has a photorealistic movement corresponding to movement behaviors of the individual, and

wherein the multi-dimensional puppet comprises: a 3D rig having a shape corresponding to at least a shape of a head and neck of the individual; a neutral layer corresponding to a look and color of at least the face and the neck of the individual; a core region overlay layer with 2D bitmaps for portions of the face and the neck of the individual; and a specular overlay layer that reproduces specular highlights of the individual;

receiving an input corresponding to a user mood and corresponding to user spatial manipulation of or interaction with the multi-dimensional puppet, wherein the input does not explicitly indicate the user mood;

determining, using at least a behavioral agent in the group of behavioral agents, the user mood based at least in part on the input; and

providing, based at least in part on the predetermined parameters, the configuration information, the group of behavioral agents, the determined user mood, and the input, the dynamic virtual representation comprising a revised multi-dimensional puppet having the one or more attributes.

19. The method of claim 18 , wherein the photorealistic movement is indistinguishable, to a human, from the movement behaviors or mannerisms of the individual.

20. The method of claim 18 , wherein the rendering is based at least in part on one or more of: characterization of movement a neck, a mouth and eyes of the individual, phonemes associated with speech of the individual, or deformations of a face of the individual while performing the movement behaviors.