Online modeling for real-time facial animation

1. A method for real-time facial animation, comprising:

providing a dynamic expression model that includes a plurality of blendshapes;

receiving tracking data from a plurality of frames in a temporal sequence, the tracking data corresponding to facial expressions of a user;

estimating one or more weight values of the blendshape for each of the plurality of frames, wherein the weight values are based on a combined 2D/3D registration of the dynamic expression model; and

generating a graphical representation of a particular facial expression of the user based on the estimated weight values of the blendshapes for the plurality of frames.

2. The method of claim 1 , further comprising:

rendering an animated visual representation of the user based on the graphical representation.

3. The method of claim 1 further comprising:

receiving further tracking data corresponding to facial expressions of the user;

estimating updated weight values based on the further tracking data; and

refining the dynamic expression model based on the estimated updated weight values.

4. The method of claim 1 , further comprising:

receiving neutral tracking data corresponding to a neutral facial expression of the user; and

initializing the dynamic expression model using the neutral tracking data.

5. The method of claim 1 , wherein the plurality of blendshapes each correspond to a unique facial expression.

6. The method of claim 1 , wherein the dynamic expression model further includes a template blendshape model, and wherein the method further comprises:

approximating one or more of the plurality of blendshapes based on the template blendshape model.

7. The method of claim 1 , wherein estimating weight values is repeated based on each of the plurality of frames in the temporal sequence.

8. A non-transitory computer readable medium comprising computer readable code for real-time facial animation, executable by one or more processors to:

provide a dynamic expression model that includes a plurality of blendshapes;

receive tracking data from a plurality of frames in a temporal sequence, the tracking data corresponding to facial expressions of a user;

estimate one or more weight values of the blendshape for each of the plurality of frames, wherein the weight values are based on a combined 2D/3D registration of the dynamic expression model; and

generate a graphical representation of a particular facial expression of the user based on the estimated weight values of the blendshapes for the plurality of frames.

9. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to:

render an animated visual representation of the user based on the graphical representation.

10. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to:

receive further tracking data corresponding to facial expressions of the user;

estimate updated weight values based on the further tracking data; and

refine the dynamic expression model based on the estimated updated weight values.

11. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to:

receive neutral tracking data corresponding to a neutral facial expression of the user; and

initialize the dynamic expression model using the neutral tracking data.

12. The non-transitory computer readable medium of claim 8 , wherein the plurality of blendshapes each correspond to a unique facial expression.

13. The non-transitory computer readable medium of claim 8 , wherein the dynamic expression model further includes a template blendshape model, and wherein the computer readable code further comprises computer readable code to:

approximate one or more of the plurality of blendshapes based on the template blendshape model.

14. The non-transitory computer readable medium of claim 8 , wherein the computer readable code to estimate weight values is repeated based on each of the plurality of frames in the temporal sequence.

15. A system for real-time facial animation, comprising:

one or more processors; and

one or more storage devices coupled to the one or more processors, and comprising computer readable code executable by the one or more processors to:

provide a dynamic expression model that includes a plurality of blendshapes;

receive tracking data from a plurality of frames in a temporal sequence, the tracking data corresponding to facial expressions of a user;

estimate one or more weight values of the blendshape for each of the plurality of frames, wherein the weight values are based on a combined 2D/3D registration of the dynamic expression model; and

generate a graphical representation of a particular facial expression of the user based on the estimated weight values of the blendshapes for the plurality of frames.

16. The system of claim 15 , further comprising computer readable code to:

render an animated visual representation of the user based on the graphical representation.

17. The system of claim 15 , further comprising computer readable code to:

receive further tracking data corresponding to facial expressions of the user;

estimate updated weight values based on the further tracking data; and

refine the dynamic expression model based on the estimated updated weight values.

18. The system of claim 15 , further comprising computer readable code to:

receive neutral tracking data corresponding to a neutral facial expression of the user; and

initialize the dynamic expression model using the neutral tracking data.

19. The system of claim 15 , wherein the plurality of blendshapes each correspond to a unique facial expression.

20. The system of claim 15 , wherein the dynamic expression model further includes a template blendshape model, and wherein the computer readable code further comprises computer readable code to:

approximate one or more of the plurality of blendshapes based on the template blendshape model.