Training system and methods for dynamically injecting expression information into an animated facial mesh
A system and method for modifying facial animations to include expression and microexpression information is disclosed. Particularly, a system and method for applying actor-generated expression data to a facial animation, either in realtime or in storage is disclosed. Present embodiments may also be incorporated into a larger training program, designed to train users to recognize various expressions and microexpressions.
1. A system for training a user to recognize facial expressions from animations of computer-simulated faces, the system comprising:
a storage system configured to store a plurality of facial expression overlays, each facial expression overlay corresponding to an emotion;
a computer system configured to:
receive user input indicating an interview question,
select a facial expression overlay from the plurality of facial expression overlays as a response to the interview question, and
combine the selected facial expression overlay into a facial animation to form a combined animation by mapping at least a portion of the selected facial expression overlay to at least a portion of the facial animation; and
a graphical user interface configured to display the combined animation.
2. The system of claim 1 , wherein the received user input is a selection of the interview question from a plurality of predefined interview questions.
3. The system of claim 1 , wherein the plurality of facial expression overlays comprise facial microexpression overlays, each facial microexpression overlay being less than 0.5 seconds in duration.
4. The system of claim 1 , wherein the computer system is further configured to receive a second user input indicating a user-identified emotion for the displayed combined animation.
5. The system of claim 4 , wherein the computer system is further configured to compare the user-identified emotion to the emotion corresponding to the selected facial expression overlay.
6. The system of claim 5 , wherein the computer system is further configured to display an indication of a result of the comparison in a user interface.
7. A method for combining a facial expression of a user into a facial animation, the method comprising:
selecting an interview question;
communicating the interview question to the user;
capturing the facial expression of the user;
determining an emotion associated with the captured facial expression;
combining a facial expression overlay that corresponds to the emotion into the facial animation to form a combined animation by mapping at least a portion of the facial expression overlay to at least a portion of the facial animation; and
displaying the combined animation.
8. The method of claim 7 , further comprising:
identifying the facial expression overlay that corresponds to the emotion from a plurality of facial expression overlays.
9. The method of claim 7 , further comprising:
converting the captured facial expression of the user into a digital animation; and
generating the facial expression overlay that corresponds to the emotion based on the converted digital animation.
10. The method of claim 9 , wherein the facial expression overlay is generated by subtracting a digital image of a non-expressive facial pose of the user from the converted digital animation.
11. The method of claim 7 , wherein the interview question is communicated to the user by generating an audible speech.
12. The method of claim 7 , further comprising:
displaying an animated texture bump map on the combined animation.
13. The method of claim 12 , wherein the animated texture bump map displays a portion of a facial expression represented by the facial expression overlay.
14. The method of claim 12 , wherein the animated texture bump map is extracted from the captured facial expression of the user.
15. A non-transitory computer-readable medium, having instructions stored therein, which when executed cause a computer to perform a set of operations comprising:
receiving user input indicating an interview question;
selecting a facial microexpression overlay from a plurality of facial microexpression overlays as a response to the interview question, each of the plurality of facial microexpression overlays corresponding to an emotion;
combining the selected facial microexpression overlay into a facial animation to form a combined animation by mapping at least a portion of the selected facial microexpression overlay to at least a portion of the facial animation; and
displaying the combined animation.
16. The non-transitory computer-readable medium of claim 15 , wherein the facial microexpression overlay is less than 0.25 seconds in duration.
17. The non-transitory computer-readable medium of claim 15 , having further instructions stored therein, which when executed cause the computer to perform a set of operations comprising:
adding a vertex displacement of the selected facial microexpression overlay with a vertex displacement of the facial animation, wherein the facial animation and each of the plurality of facial microexpression overlays comprise vertex displacements over time.
18. The non-transitory computer-readable medium of claim 15 , wherein the selected facial microexpression overlay is mapped to a digital mesh of the facial animation.
19. The non-transitory computer-readable medium of claim 18 , wherein the digital mesh of the facial animation uses a spring-based, dermal elasticity model.
20. The non-transitory computer-readable medium of claim 15 , having further instructions stored therein, which when executed cause the computer to perform a set of operations comprising:
selecting a second facial microexpression overlay from the plurality of facial microexpression overlays as a response to the interview question; and
combining the second facial microexpression overlay into the facial animation such that the selected facial microexpression overlay and the second facial microexpression overlay appear simultaneously in the combined animation.