IP Library › Granted Patent US 11,587,279
Granted Patent B2
US 11,587,279 · App. 17/682,862 · Granted Feb 21, 2023

Augmented facial animation

Inventors: Yikai Fang (Beijing, CN); Yangzhou Du (Beijing, CN); Qiang Eric Li (Beijing, CN); Xiaofeng Tong (Beijing, CN); Wenlong Li (Beijing, CN); Minje Park (Seongnam, KR); Myung-Ho Ju (Seoul, KR); Jihyeon Kate Yi (Yongin-si, KR); Tae-Hoon Pete Kim (Seoul, KR)
Assignee: INTEL CORPORATION
G06T13/40G06T5/002G06V40/166G06V40/176G06V40/28G06T2207/10016G06T2207/20092G06T2207/20182G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,587,279
App. No.
17/682,862
Granted
Feb 21, 2023
Kind
B2
Abstract

Examples of systems and methods for augmented facial animation are generally described herein. A method for mapping facial expressions to an alternative avatar expression may include capturing a series of images of a face, and detecting a sequence of facial expressions of the face from the series of images. The method may include determining an alternative avatar expression mapped to the sequence of facial expressions, and animating an avatar using the alternative avatar expression.

Claims (47)

1. At least one storage device comprising instructions that, when executed, cause processor circuitry to:

identify an augmented facial appearance based on user input, the augmented facial appearance to be presented on a display based on one or more images of a face captured by a camera, the augmented facial appearance corresponding to an expression beyond an ability of a human to express;

detect a particular movement of the face in successive ones of the one or more images; and

in response to detection of the particular movement, generate augmented images based on the successive images, the augmented images including the augmented facial appearance.

2. The at least one storage device of claim 1 , wherein the particular movement includes a mouth opening.

3. The at least one storage device of claim 1 , wherein the augmented facial appearance includes a mouth that is larger than humanly possible relative to other facial features in the augmented facial appearance.

4. The at least one storage device of claim 1 , wherein the instructions cause the processor circuitry to cause the camera to capture the one or more images of the face.

5. The at least one storage device of claim 1 , wherein the instructions cause the processor circuitry to detect a hand in the one or more images.

6. The at least one storage device of claim 1 , wherein the instructions cause the processor circuitry to detect the particular movement based on detection of movement of a particular landmark point on the face.

7. The at least one storage device of claim 6 , wherein the instructions cause the processor circuitry to detect the movement of the particular landmark point using dynamic time warping.

8. The at least one storage device of claim 1 , wherein the augmented facial appearance is one of a plurality of augmented facial appearances, different ones of the plurality of augmented facial appearances selectable by a user.

9. A method comprising:

identifying an augmented facial animation based on user input, the augmented facial animation to be presented on a display based on one or more images of a face captured by a camera, the augmented facial animation including an appearance beyond an ability of a human face to replicate;

detecting a particular movement of the face in successive ones of the one or more images; and

in response to detection of the particular movement, generating, by executing an instruction with processor circuitry, augmented images based on the successive images, the augmented images including the augmented facial animation.

10. The method of claim 9 , wherein the particular movement includes a mouth opening.

11. The method of claim 9 , wherein the augmented facial animation includes a mouth that is larger than humanly possible relative to other facial features in the augmented facial animation.

12. The method of claim 9 , further including causing the camera to capture the one or more images of the face.

13. The method of claim 9 , further including detecting a hand in the one or more images.

14. The method of claim 9 , further including detecting the particular movement based on detection of movement of a particular landmark point on the face.

15. The method of claim 14 , further including detecting the movement of the particular landmark point using dynamic time warping.

16. The method of claim 9 , wherein the augmented facial animation is one of a plurality of augmented facial animations, different ones of the plurality of augmented facial animations selectable by a user.

17. At least one storage device comprising instructions that, when executed, cause processor circuitry to:

identify a facial feature to be changed based on user input, the changed facial feature to be presented on a display based on one or more images of a face captured by a camera, the facial feature to be changed to produce a surreal facial appearance;

detect a particular movement of the face in successive ones of the one or more images; and

in response to detection of the particular movement, generate updated images based on the successive images, the updated images including the surreal facial appearance.

18. The at least one storage device of claim 17 , wherein the particular movement includes a mouth opening.

19. The at least one storage device of claim 17 , wherein the surreal facial appearance corresponds to a mouth that is larger than humanly possible relative to other facial features.

20. The at least one storage device of claim 17 , wherein the processor circuitry is to cause the camera to capture the one or more images of the face.

21. The at least one storage device of claim 17 , wherein the processor circuitry is to detect a hand in the one or more images.

22. The at least one storage device of claim 17 , wherein the processor circuitry is to detect the particular movement based on detection of movement of a particular landmark point on the face.

23. The at least one storage device of claim 22 , wherein the processor circuitry is to detect the movement of the particular landmark point using dynamic time warping.

24. The at least one storage device of claim 17 , wherein the facial feature to be changed is one of a plurality of facial features to be changed, different ones of the plurality of facial features selectable by a user.

25. An apparatus comprising

memory;

instructions; and

processor circuitry to execute the instructions to:

identify an augmented facial image based on user input, the augmented facial image to be presented on a display based on one or more images of a face captured by a camera, the augmented facial image corresponding to a surreal facial appearance;

detect a particular movement of the face in successive ones of the one or more images; and

in response to detection of the particular movement, generate updated images based on the successive images, the updated images including the surreal facial appearance.

26. The apparatus of claim 25 , wherein the particular movement includes a mouth opening.

27. The apparatus of claim 25 , wherein the surreal facial appearance corresponds to a mouth that is larger than humanly possible relative to other facial features.

28. The apparatus of claim 25 , wherein the processor circuitry is to cause the camera to capture the one or more images of the face.

29. The apparatus of claim 25 , wherein the processor circuitry is to detect a hand in the one or more images.

30. The apparatus of claim 25 , wherein the processor circuitry is to detect the particular movement based on detection of movement of a particular landmark point on the face.

31. The apparatus of claim 30 , wherein the processor circuitry is to detect the movement of the particular landmark point using dynamic time warping.

32. The apparatus of claim 25 , wherein the surreal facial appearance is one of a plurality of surreal facial appearances, different ones of the plurality of surreal facial appearances selectable by a user.

Continuity (5)
Continuation 16987707 · Aug 7, 2020
Continuation 16172664 · Oct 26, 2018
Continuation 15822271 · Nov 27, 2017
Continuation 14779501
Related Publication 20220237845A1 · Jul 28, 2022