Online modeling for real-time facial animation

1. A method for real-time facial animation, comprising:

providing a dynamic expression model that includes a plurality of blendshapes;

receiving tracking data corresponding to a facial expression of a user;

estimating tracking parameters based on the dynamic expression model and the tracking data; and

refining the dynamic expression model based on the tracking data and the estimated tracking parameters when one or more of the plurality of blendshapes of the dynamic expression model have a coverage coefficient that satisfies a specified criteria, to produce a refined dynamic expression model.

2. The method of claim 1 , wherein a blendshape's coverage coefficient is indicative of how applicable the tracking data is for the blendshape.

3. The method of claim 1 , wherein:

the specified criteria comprises a threshold; and

the coverage coefficient satisfies the specified criteria when a value of the coverage coefficient is less than the threshold.

4. The method of claim 1 , wherein estimating tracking parameters and refining the dynamic expression model are performed in real-time.

5. The method of claim 1 , further comprising generating a graphical representation corresponding to the facial expression of the user based on the tracking parameters.

6. The method of claim 1 , wherein the plurality of blendshapes comprise a neutral model and one or more additional models, each additional model indicative of a basic facial expression, and each of the plurality of blendshapes representative of a three-dimensional ( 3 D) mesh model.

7. The method of claim 6 , wherein refining the dynamic expression model further includes applying at least one corrective deformation field to at least some of the plurality of blendshapes.

8. The method of claim 7 , wherein the neutral model includes a blendshape b o representing a neutral facial expression and the dynamic expression model further includes an identity model, and wherein the method further comprises matching the blendshape b o representing the neutral facial expression to a neutral expression of the user based on the identity model.

9. The method of claim 1 , further comprising:

Receiving tracking data corresponding to a neutral facial expression of the user; and

initializing the dynamic expression model using the tracking data corresponding to the neutral facial expression of the user.

10. The method of claim 1 , wherein refining the dynamic expression model based on the tracking data and the estimated tracking parameters comprises refining the dynamic expression model based on the tracking data, the estimated tracking parameters, and aggregated tracking data, wherein the aggregated tracking data is based on additional received tracking data corresponding to one or more past facial expressions of the user.

11. The method of claim 1 , further comprising:

automatically creating a virtual avatar of the user based on the dynamic expression model; and

generating a graphical representation corresponding to the facial expression of the user based on the virtual avatar.

12. A processing device, comprising:

an input interface configured to receive tracking data corresponding to a facial expression of a user;

a memory configured to store a dynamic expression model that includes a plurality of blendshapes; and

a processing component coupled to the input interface and the memory, wherein the processing component is configured to:

estimate tracking parameters based on the dynamic expression model and the tracking data; and

refine the dynamic expression model based on the tracking data and the estimated tracking parameters when one or more of the plurality of blendshapes of the dynamic expression model have a coverage coefficient that satisfies a specified criteria, to produce a refined dynamic expression model.

13. The device of claim 12 , wherein a blendshape's coverage coefficient is indicative of how applicable the tracking data is for the blendshape.

14. The device of claim 12 , wherein the plurality of blendshapes comprise a neutral model and one or more additional models, each additional model indicative of a basic facial expression, and each of the plurality of blendshapes representative of a three-dimensional ( 3 D) mesh model.

15. The device of claim 14 , wherein the processing component is further configured to apply at least one corrective deformation field to at least some of the plurality of blendshapes.

16. The device of claim 12 , wherein the processing component is further configured to:

receive tracking data corresponding to a neutral facial expression of the user; and

initialize the dynamic expression model using the tracking data corresponding to the neutral facial expression of the user.

17. The device of claim 12 , wherein the processing component is further configured to:

automatically create a virtual avatar of the user based on the dynamic expression model; and

generate a graphical representation corresponding to the facial expression of the user based on the virtual avatar.

18. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

provide a dynamic expression model that includes a plurality of blendshapes;

receive tracking data corresponding to a facial expression of a user;

estimate tracking parameters based on the dynamic expression model and the tracking data; and

refine the dynamic expression model based on the tracking data and the estimated tracking parameters when one or more of the plurality of blendshapes of the dynamic expression model have a coverage coefficient that satisfies a specified criteria, to produce a refined dynamic expression model.

19. The non-transitory program storage device of claim 18 , wherein a blendshape's coverage coefficient is indicative of how applicable the tracking data is for the blendshape.

20. The non-transitory program storage device of claim 18 , wherein the plurality of blendshapes comprise a neutral facial model and one or more additional models, each additional model indicative of a basic facial expression, and each of the plurality of blendshapes representative of a three-dimensional (3D) mesh model.

21. The non-transitory program storage device of claim 20 , further comprising instructions to apply at least one corrective deformation field to at least some of the plurality of blendshapes.

22. The non-transitory program storage device of claim 20 , further comprising instructions to:

receive tracking data corresponding to the neutral facial expression of the user; and

initialize the dynamic expression model using the tracking data corresponding to the neutral facial expression of the user.

23. The non-transitory program storage device of claim 18 , wherein the instructions to refine the dynamic expression model comprise instructions to refine the dynamic expression model based on the tracking data, the estimated tracking parameters, and aggregated tracking data, wherein the aggregated tracking data is based on additional received tracking data corresponding to one or more past facial expressions of the user.

24. The non-transitory program storage device of claim 18 , further comprising instructions to:

automatically create a virtual avatar of the user based on the dynamic expression model; and

generate a graphical representation corresponding to the facial expression of the user based on the virtual avatar.