Systems and methods for improving animation of computer-generated avatars

1. A computer-implemented method comprising:

identifying an action unit associated with a face of a user by:

capturing a set of images of the face of the user producing a viseme while the user speaks a sound; and

associating at least one facial muscle engaged by the user to produce the viseme based on the set of images and the sound, such that the facial muscle is associated with the action unit;

training an artificial neural network to determine action unit parameters corresponding to the user, the action unit, and the viseme based on the set of images of the user producing the viseme, the action unit parameters comprising:

an onset curve that describes a rate of change over time of an intensity level of the facial muscle as the user produces the viseme, the intensity level of the facial muscle describing a degree of transition of the facial muscle between a neutral position and a maximum position; and

a falloff curve that describes a rate of change over time of the intensity level of the facial muscle subsequent to the user producing the viseme;

determining, by providing the set of images as input to the trained artificial neural network, a set of action unit parameters associated with the user, the action unit, and the viseme, the set of action unit parameters comprising:

an onset curve corresponding to the user, the action unit, and the viseme; and

a falloff curve corresponding to the user, the action unit, and the viseme;

detecting that the user has produced the sound; and

directing, in response to detecting that the user has produced the sound, a computer-generated avatar that models the action unit to produce the viseme by adjusting the modeled action unit over time in accordance with the onset curve and the falloff curve.

2. The computer-implemented method of claim 1 , wherein:

the set of action unit parameters further comprises:

a target intensity level associated with the viseme; and

a source intensity level associated with a previous viseme;

the onset curve describes a rate of change over time of the intensity level of user engagement of the facial muscle from the source intensity level to the target intensity level; and

the falloff curve describes a rate of change over time of the intensity level of user engagement of the facial muscle from the target intensity level to an additional intensity level.

3. The computer-implemented method of claim 1 , further comprising:

identifying an additional action unit associated with at least one facial muscle engaged by the user to produce an additional viseme associated with an additional sound; determining:

an additional onset curve that describes a change of the action unit over time to produce the additional viseme; and

an additional falloff curve that describes a change of the action unit over time subsequent to producing the additional viseme;

detecting the user producing the additional sound subsequent to the user producing the sound; and

transitioning the computer-generated avatar from producing the viseme to producing the additional viseme in accordance with the falloff curve and the additional onset curve in response to detecting the user producing the additional sound subsequent to the user producing the sound.

4. The computer-implemented method of claim 3 , wherein:

the set of action unit parameters further comprises a peak transition velocity of the facial muscle; and

transitioning the computer-generated avatar from producing the viseme to producing the additional viseme comprises transitioning the computer-generated avatar from producing the viseme to producing the additional viseme such that a transition velocity of a portion of the computer-generated avatar that corresponds to the facial muscle remains below the peak transition velocity of the facial muscle during the transition.

5. The computer-implemented method of claim 1 , wherein:

the sound comprises a set of phonemes;

identifying the action unit associated with the face of the user further comprises:

recording audio of the user as the user speaks the set of phonemes;

identifying the viseme based on the set of images and the recorded audio; and

associating a feature of the face of the user with the action unit based on the identification of the viseme, the set of images, and the recorded audio; and

determining the set of action unit parameters associated with the action unit and the viseme further comprises determining a set of action unit parameters associated with the action unit, the viseme, and the feature of the face of the user based on the recorded audio.

6. The computer-implemented method of claim 1 , further comprising generating a digital signature of the face of the user based on the set of action unit parameters associated with the action unit and the viseme.

7. The computer-implemented method of claim 6 , wherein directing the computer-generated avatar that represents the user to produce the viseme in accordance with the set of action unit parameters associated with the action unit in response to detecting that the user has produced the sound comprises directing the computer-generated avatar that represents the user to produce the viseme in accordance with the digital signature of the face of the user.

8. The computer-implemented method of claim 1 , wherein the action unit comprises at least one of:

a Facial Action Coding System (FACS) action unit; or

an FACS action descriptor.

9. The computer-implemented method of claim 1 , wherein the computer-generated avatar that represents the user comprises at least one of:

an augmented reality avatar presented within an augmented reality environment;

a virtual reality avatar presented within a virtual reality environment; or

a video conferencing avatar presented within a video conferencing application.

10. The computer-implemented method of claim 1 , wherein:

the method further comprises training the artificial neural network to identify action units associated with muscle groups engaged by user faces based on sounds produced by users; and

identifying the action unit associated with the face of the user further comprises providing the artificial neural network with the sound produced by the user.

11. A system comprising:

an identifying module, stored in memory, that identifies an action unit associated with a face of a user by:

capturing a set of images of the face of the user producing a viseme while the user speaks a sound; and

associating at least one facial muscle engaged by the user to produce the viseme based on the set of images and the sound, such that the facial muscle is associated with the action unit;

a determining module, stored in memory, that:

trains an artificial neural network to determine action unit parameters corresponding to the user, the action unit, and the viseme based on the set of images of the user producing the viseme, the action unit parameters comprising:

an onset curve that describes a rate of change over time of an intensity level of the facial muscle as the user produces the viseme, the intensity level of the facial muscle describing a degree of transition of the facial muscle between a neutral position and a maximum position; and

a falloff curve that describes a rate of change over time of the intensity level of the facial muscle subsequent to the user producing the viseme; and

determines, by providing the set of images as input to the trained artificial neural network, a set of action unit parameters associated with the user, the action unit, and the viseme, the set of action unit parameters comprising:

an onset curve corresponding to the user, the action unit, and the viseme; and

a falloff curve corresponding to the user, the action unit, and the viseme;

a detecting module, stored in memory, that detects that the user has produced the sound;

a directing module, stored in memory, that directs, in response to detecting that the user has produced the sound, a computer-generated avatar that models the action unit to produce the viseme by adjusting the modeled action unit over time in accordance with the onset curve and the falloff curve; and

at least one physical processor that executes the identifying module, the determining module, the detecting module, and the directing module.

12. The system of claim 11 , wherein:

the set of action unit parameters further comprises:

a target intensity level associated with the viseme; and

a source intensity level associated with a previous viseme;

the onset curve describes a rate of change over time of the intensity level of user engagement of the facial muscle from the source intensity level to the target intensity level; and

the falloff curve describes a rate of change over time of the intensity level of user engagement of the facial muscle from the target intensity level to an additional intensity level.

13. The system of claim 11 , wherein:

the identifying module further identifies an additional action unit associated with at least one facial muscle engaged by the user to produce an additional viseme associated with an additional sound;

the determining module further determines:

an additional onset curve that describes a change of the action unit over time to produce the additional viseme; and

an additional falloff curve that describes a change of the action unit over time subsequent to producing the additional viseme;

the detecting module further detects the user producing the additional sound subsequent to the user producing the sound; and

the directing module further transitions the computer-generated avatar from producing the viseme to producing the additional viseme in accordance with the falloff curve and the additional onset curve in response to detecting the user producing the additional sound subsequent to the user producing the sound.

14. The system of claim 13 , wherein:

the set of action unit parameters further comprises a peak transition velocity of the facial muscle; and

the directing module further transitions the computer-generated avatar from producing the viseme to producing the additional viseme by transitioning the computer-generated avatar from producing the viseme to producing the additional viseme such that a transition velocity of a portion of the computer-generated avatar that corresponds to the facial muscle remains below the peak transition velocity of the facial muscle during the transition.

15. The system of claim 11 , wherein:

the sound comprises a set of phonemes;

the identifying module identifies the action unit associated with the face of the user by further:

recording audio of the user as the user speaks the set of phonemes;

identifying the viseme based on the set of images and the recorded audio; and

associating a feature of the face of the user with the action unit based on the identification of the viseme, the set of images, and the recorded audio; and

the determining module determines the set of action unit parameters associated with the action unit and the viseme by further determining a set of action unit parameters associated with the action unit, the viseme, and the feature of the face of the user based on the recorded audio.

16. The system of claim 11 , wherein the determining module further determines the set of action unit parameters associated with the action unit and the viseme by generating a digital signature of the face of the user based on the set of action unit parameters associated with the action unit and the viseme.

17. The system of claim 16 , wherein the directing module directs the computer-generated avatar that represents the user to produce the viseme in accordance with the set of action unit parameters associated with the action unit in response to detecting that the user has produced the sound by directing the computer-generated avatar that represents the user to produce the viseme in accordance with the digital signature of the face of the user.

18. The system of claim 11 , wherein the computer-generated avatar that represents the user comprises at least one of:

an augmented reality avatar presented within an augmented reality environment;

a virtual reality avatar presented within a virtual reality environment; or

a video conferencing avatar presented within a video conferencing application.

19. The system of claim 11 , wherein:

the determining module further trains the artificial neural network to identify action units associated with muscle groups engaged by user faces based on sounds produced by users; and

the identifying module identifies the action unit associated with the face of the user by further providing the artificial neural network with the sound produced by the user.

20. A non-transitory computer-readable medium comprising computer-readable instructions that, when executed by at least one processor of a computing system, cause the computing system to:

identify an action unit associated with a face of a user by:

capturing a set of images of the face of the user producing a viseme while the user speaks a sound; and

associating at least one facial muscle engaged by the user to produce the viseme based on the set of images and the sound, such that the facial muscle is associated with the action unit;

train an artificial neural network to determine action unit parameters corresponding to the user, the action unit, and the viseme based on the set of images of the user producing the viseme, the action unit parameters comprising:

an onset curve that describes a rate of change over time of an intensity level of the facial muscle as the user produces the viseme, the intensity level of the facial muscle describing a degree of transition of the facial muscle between a neutral position and a maximum position; and

a falloff curve that describes a rate of change over time of the intensity level of the facial muscle subsequent to the user producing the viseme;

determine, by providing the set of images as input to the trained artificial neural network, a set of action unit parameters associated with the user, the action unit, and the viseme, the set of action unit parameters comprising:

an onset curve corresponding to the user, the action unit, and the viseme; and

a falloff curve corresponding to the user, the action unit, and the viseme;

detect that the user has produced the sound; and

direct, in response to detecting that the user has produced the sound, a computer-generated avatar that models the action unit to produce the viseme by adjusting the modeled action unit over time in accordance with the onset curve and the falloff curve.