Capturing and processing facial motion data

1. A method of capturing and processing motion data of a facial surface of an actor, comprising:

projecting speckle patterns on the facial surface of the actor using near-IR light to enable triangulation;

capturing frame by frame images of the speckle pattern applied on the facial surface of the actor using at least one motion capture camera disposed on a head-mounted system by tracking and analyzing movement of the speckle pattern;

performing, in the head-mounted system, a tracking function on said frame by frame images of said plurality of sensors to accurately map said plurality of sensors for each frame; and

generating, in the head-mounted system, a modeled surface representing the facial surface of the actor by triangulating the speckle patterns.

2. The method of claim 1 , wherein projecting speckle patterns on the facial surface of the actor comprises

forming a grid of horizontal and vertical lines or curves over the facial surface of the actor.

3. The method of claim 2 , wherein tracking and analyzing movement of the pattern comprises

tracking and analyzing the movement of the horizontal and vertical lines or curves and vertices formed by intersections of the grid in the pattern.

4. The method of claim 1 , further comprising

performing 2-D to 3-D triangulation after said capturing frame by frame images of the pattern.

5. The method of claim 4 , further comprising

removing perspective distortion after the frame by frame images of the pattern are captured.

6. The method of claim 5 , wherein said removing perspective distortion includes:

unwarping the frame by frame images; and

improving the 2-D to 3-D triangulation.

7. The method of claim 1 , wherein tracking and analyzing movement of the pattern comprises

tracking and analyzing the movement of lines and vertices formed by grid intersection in the pattern.

8. The method of claim 1 , wherein projecting speckle patterns on the facial surface of the actor comprises

passing an invisible strip of light over the facial surface of the actor using a laser.

9. The method of claim 8 , wherein capturing frame by frame images of the pattern applied on the facial surface of the actor comprises:

scanning strip contours of the pattern of the strip of light using said at least one motion capture camera;

reconstructing lines created by the strip contours as surface geometry; and

joining the lines to generate a mesh.

10. The method of claim 1 , wherein projecting speckle patterns on the facial surface of the actor comprises

applying a pattern of physical material to the facial surface of the actor.

11. The method of claim 10 , wherein applying a pattern of physical material comprises

applying or spraying a make-up, ink, paint, or dye onto the facial surface of the actor.

12. The method of claim 10 , wherein applying a pattern of physical material comprises:

printing or silk screening the pattern to the physical material; and

attaching the physical material to the facial surface of the actor.

13. The method of claim 10 , wherein applying a pattern of physical material comprises

applying piezoelectric strips on the facial surface,

wherein the piezoelectric strips measure resistance in bending of the strips with movement of the facial surface.

14. A system, comprising:

at least one motion capture camera configured to capture motion data from a facial surface of an actor using a plurality of sensors disposed on the facial surface of the actor; and

a motion capture processor coupled to said at least one motion capture camera, said motion capture processor including:

a storage unit;

an image capture module configured to control said at least one motion capture camera to capture frame by frame images of said plurality of sensors disposed on the facial surface of the actor, said image capture module receiving and storing said frame by frame images on said storage unit;

a sensor tracking module configured to retrieve the stored frame by frame images from said storage unit, to perform a tracking function on the images to accurately map said plurality of sensors for each frame, and to provide said motion data from the facial surface;

a digital modeling module configured to receive said motion data to generate a modeled surface representing the facial surface of the actor;

a microphone configured to record audio from the actor; and

a cooling unit configured to cool the actor, wherein the cooling unit is turned off when the microphone is turned on.

15. A method of capturing and processing motion data of a facial surface of an actor, comprising:

applying speckle patterns on a set of interest points of the facial surface of the actor using reflective material,

wherein the set of interest points is a set of feature points chosen by scanning and modeling facial feature of the actor;

capturing frame by frame images of the speckle patterns applied on the set of interest points of the facial surface of the actor using at least one motion capture camera disposed on a head-mounted system by tracking and analyzing movement of the speckle patterns;

performing, in the head-mounted system, a tracking function on said frame by frame images of said plurality of sensors to accurately map said plurality of sensors for each frame; and

generating, in the head-mounted system, a 3-D mesh of a modeled surface representing the set of interest points of the facial surface of the actor by triangulating the speckle patterns.