Method and apparatus for providing natural facial animation

1. An apparatus for natural inter-viseme animation of a 3D model driven by speech recognition, comprising:

a speech recognition engine that identifies phonemes in a sequence of speech; and

a processor programmed to associate the phonemes with corresponding visemes which represent facial expressions in said 3D model;

said processor programmed to determine and limit any of velocity and acceleration of a normalized parameter vector, the velocity or acceleration occurring in movement from a first one of the visemes to a second one of the visemes;

said processor programmed to establish and apply a maximum value to the velocity or acceleration to limit the velocity or acceleration, wherein as a distance between the first viseme and the second viseme decreases, the velocity or acceleration decreases from the maximum value to zero.

2. The apparatus of claim 1 , said processor programmed to map each element of said parameter vector to animation node outputs of said 3D model based on mesh blending, weighted by a mix of key frames, to effect inter-viseme transitions.

3. The apparatus of claim 1 , wherein said processor is configured to generate the limit based on a maximum distance between two of the visemes, the maximum distance being a maximum value of distances between two of the visemes.

4. A computer implemented method for facial animation, comprising the computer implemented steps of:

receiving phonemes in a sequence of speech from a speech recognition engine;

associating at least two of said phonemes with corresponding visemes which represent facial expressions;

defining a viseme weighting parameter vector in which each element stands for a weight of one viseme;

applying said viseme weighting parameter vector to each viseme to limit any of a velocity and acceleration component in connection with a transition from a first one of the visemes to a second one of the visemes; and

establishing and applying a maximum value to the velocity component or acceleration component to limit the velocity component or acceleration component, wherein as a distance between the first viseme and the second viseme decreases, the velocity component or acceleration component decreases from the maximum value to zero.

5. The method of claim 4 , further comprising the step of:

animating a face with said viseme weighting parameter vector.

6. The method of claim 4 , further comprising the step of:

representing each element as a weight for a morph when mesh-morphing is used for facial animation.

7. The method of claim 4 , further comprising the step of:

representing each element as a weight for a plurality of parameters to create a shape of a viseme when bone skinning is used for facial animation.

8. The method of claim 4 , further comprising the steps of:

calculating the speed of a transition between the visemes in speech recognition based facial animation; and

applying said viseme weighting parameter vector to said transition to maintain said transition speed within a weighted limit established by said viseme weighting parameter vector by any of velocity and acceleration.

9. The method of claim 4 , wherein a target facial shape comprises a mixture of more than one viseme.

10. The method of claim 4 , further comprising the step of:

displaying a viseme for each facial expression that corresponds to each recognized phoneme.

11. The method of claim 10 , wherein each said recognized phoneme has a corresponding one of the following visemes:

/a/, /i/, /u/, /e/, /o/, closed mouth (C), and common consonant (CC).

12. The method of claim 4 , further comprising the steps of:

determining any of a velocity and an acceleration weighting; and

applying said weighting as a transition value from a current viseme to a target viseme on a frame by frame basis.

13. The method of claim 12 , said acceleration value comprising a weighting for an inter-viseme transition.

14. The method of claim 4 , further comprising the step of:

using relative Euclid distance to calculate a vector distance between any two of the visemes for any of said velocity component and said acceleration component.

15. The method of claim 4 , further comprising the step of:

adding hysteresis to a viseme-to-viseme transition with said viseme weighting parameter vector to avoid an influence of unwanted phonemes as a frame is created.

16. The method of claim 4 , further comprising the step of:

determining a vector division factor based upon an animation frame rate that is associated with the visemes.

17. An apparatus for effecting speech driver facial animation, comprising:

a speech recognition engine that provides a time sequence of phonemes; and

a processor that is programmed to link each phoneme within the sequence to a corresponding viseme to define a shape of a facial expression;

said processor programmed to limit any of a velocity and acceleration during a transition from a first one of the visemes to a second one of the visemes, said processor programmed to establish and apply a maximum value to the velocity or acceleration to limit the velocity or acceleration, wherein as a distance between the first viseme and the second viseme decreases, the velocity or acceleration decreases from the maximum value to zero.

18. The apparatus of claim 17 , further comprising:

said processor programmed to calculate a distance between a facial expression displayed by a current viseme and a facial expression displayed by a target viseme;

said processor programmed to generate said weighting by dividing a maximum of said distance by a fixed number which is based upon animation frame speed; and

said processor programmed to apply said weighting as any of said velocity limitation and said acceleration limitation.

19. A computer implemented method for effecting speech driver facial animation, comprising computer implemented operations of:

providing a time sequence of phonemes with speech recognition engine;

linking each phoneme within the sequence to a corresponding viseme to define a shape of a facial expression;

limiting any of a velocity and acceleration during a transition from a first one of the visemes to a second one of the visemes; and

establishing and applying a maximum value to the velocity or acceleration to limit the velocity or acceleration, wherein as a distance between the first viseme and the second viseme decreases, the velocity or acceleration decreases from the maximum value to zero.

20. The method of claim 19 , further comprising the steps of:

calculating a distance between a facial expression displayed current viseme and a facial expression displayed by a target viseme;

generating said weighting by dividing a maximum of said distance by a fixed number which is based upon animation frame speed; and

applying said weighting as any of said velocity limitation and said acceleration limitation.