IP Library Granted Patent US 11,741,682
Granted Patent B2
US 11,741,682 · App. 17/388,946 · Granted Aug 29, 2023

Face augmentation in video

Inventors: Ke Chen (Cupertino, CA); Zhipin Deng (Beijing, CN); Xiaoxia Cai (Beijing, CN); Chen Wang (Beijing, CN); Ya-Ti Peng (Sunnyvale, CA); Yi-Jen Chiu (San Jose, CA); Lidong Xu (Beijing, CN)
Assignee: Tahoe Research, Ltd.
G06T19/20G06T5/00G06T7/90G06T15/005G06V40/161G06V40/168G06T2207/10016G06T2207/10024G06T2207/20028G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,741,682
App. No.
17/388,946
Granted
Aug 29, 2023
Kind
B2
Abstract

Systems, apparatus, articles of manufacture and methods for face augmentation in video are disclosed. An example apparatus includes executable code to detect a face of a subject in the video, detect a gender of the subject based on the face, detect a skin tone of the subject based on the face, apply a first process to smooth skin on the face in the video, apply a second process to change the skin tone of the face, apply a third process to slim the face, apply a fourth process to adjust a size of eyes on the face, and apply a fifth process to remove an eye bag from the face. One or more of the first process, the second process, the third process, the fourth process, or the fifth process adjustable based on one or more of the gender or an age. The example apparatus also includes one or more processors to generate modified video with beauty effects, the beauty effects based on one or more of the first process, the second process, the third process, the fourth process, or the fifth process.

Claims (77)

1. A smartphone comprising:

a housing;

a display;

a battery;

a wireless transmitter;

a camera configured to capture images to generate a video;

one or more processors including a graphics processing unit; and

memory including instructions configured to generate modified video based on the video, the instructions including executable code configured to:

detect a face of a subject in the video,

detect a gender of the subject based on the face,

detect a skin tone of the subject based on the face by determining a skin tone likelihood score for each pixel of each of the images and performing an averaging operation on the skin tone likelihood score,

apply a first process to smooth skin on the face in the video,

apply a second process to change the skin tone of the face,

apply a third process to slim the face,

apply a fourth process to adjust a size of eyes on the face, and

apply a fifth process to remove an eye bag from the face,

one or more of the first process, the second process, the third process, the fourth process, or the fifth process being adjustable based on one or more of the gender or an age;

one or more of the one or more processors being configured to generate the modified video with beauty effects, the beauty effects based on one or more of the first process, the second process, the third process, the fourth process, or the fifth process; and

the wireless transmitter being configured to wirelessly transmit the modified video to a device different from the smartphone.

2. The smartphone of claim 1 , wherein the one or more processors is configured to create the modified video by applying the beauty effects to each of the images as the images are captured.

3. The smartphone of claim 2 , wherein the camera is configured to capture the images at at least thirty frames per second.

4. The smartphone of claim 1 , wherein one or more of the one or more processors is configured to execute gaming instructions.

5. The smartphone of claim 1 , wherein the one or more processors includes a virtual graphics processing unit.

6. The smartphone of claim 1 , wherein the one or more processors is configured to generate the modified video with beauty effects by applying two or more of the first process, the second process, the third process, the fourth process, or the fifth process simultaneously.

7. The smartphone of claim 1 , wherein

the averaging operation results in a smoothed skin score, and

each of the first process and the second process comprises:

modifying input pixels from the images to generate modified pixels, and

blending the input pixels with the modified pixels weighted based on the smoothed skin score to produce processed pixels for the modified video.

8. A smartphone comprising:

a housing;

a display;

a battery;

means for capturing images to generate a video;

means for generating modified video, the generating means being configured to:

detect a face of a subject in the video,

detect a gender of the subject based on the face,

detect a skin tone of the subject based on the face by determining a skin tone likelihood score for each pixel of each of the images and performing an averaging operation on the skin tone likelihood score,

apply a first process to smooth skin on the face in the video,

apply a second process to change the skin tone of the face,

apply a third process to slim the face,

apply a fourth process to adjust a size of eyes on the face, and

apply a fifth process to remove an eye bag from the face,

one or more of the first process, the second process, the third process, the fourth process, or the fifth process to be adjusted based on one or more of the gender or an age;

the generating means being configured to generate the modified video with beauty effects, the beauty effects corresponding to one or more of the first process, the second process, the third process, the fourth process, or the fifth process; and

means for wirelessly transmitting the modified video to a device different from the smartphone.

9. The smartphone of claim 8 , wherein the generating means is configured to create the modified video by applying the beauty effects to each of the images as the images are captured.

10. The smartphone of claim 9 , wherein the capturing means is configured to capture the images at at least thirty frames per second.

11. The smartphone of claim 8 , wherein the generating means is configured to execute gaming instructions.

12. The smartphone of claim 8 , wherein the generating means includes a virtual graphics processing unit.

13. The smartphone of claim 8 , wherein the generating means is configured to generate the modified video with beauty effects by simultaneously applying two or more of the first process, the second process, the third process, the fourth process, or the fifth process.

14. The smartphone of claim 8 , wherein

the averaging operation results in a smoothed skin score, and

each of the first process and the second process comprises:

modifying input pixels from the images to generate modified pixels, and

blending the input pixels with the modified pixels weighted based on the smoothed skin score to produce processed pixels for the modified video.

15. A computer readable storage device or storage disk comprising instructions, the instructions including executable code that, when executed, cause at least one processor to, at least:

detect a face of a subject in a video generated from captured images,

detect a gender of the subject based on the face,

detect a skin tone of the subject based on the face by determining a skin tone likelihood score for each pixel of each of the images and performing an averaging operation on the skin tone likelihood score,

apply a first process to smooth skin on the face in the video,

apply a second process to change the skin tone of the face,

apply a third process to slim the face,

apply a fourth process to adjust a size of eyes on the face, and

apply a fifth process to remove an eye bag from the face,

one or more of the first process, the second process, the third process, the fourth process, or the fifth process to proceed differently based on one or more of the gender or an age;

generate modified video with beauty effects, the beauty effects corresponding to one or more of the first process, the second process, the third process, the fourth process, or the fifth process; and

wirelessly transmit the modified video.

16. The computer readable storage device or storage disk of claim 15 , wherein the instructions, when executed, cause the at least one processor to create the modified video by applying the beauty effects to each of the images as the images are captured.

17. The computer readable storage device or storage disk of claim 16 , wherein the instructions, when executed, cause the at least one processor to cause a camera to capture the images at at least thirty frames per second.

18. The computer readable storage device or storage disk of claim 15 , wherein the instructions include gaming instructions.

19. The computer readable storage device or storage disk of claim 15 , wherein the instructions, when executed, cause the at least one processor to simultaneously apply two or more of the first process, the second process, the third process, the fourth process, or the fifth process.

20. The computer readable storage device or storage disk of claim 15 , wherein

the averaging operation results in a smoothed skin score, and

each of the first process and the second process comprises:

modifying input pixels from the images to generate modified pixels, and

blending the input pixels with the modified pixels weighted based on the smoothed skin score to produce processed pixels for the modified video.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2022
From: INTEL CORPORATION
To: TAHOE RESEARCH, LTD.
Reel/Frame 061175/0176 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 12, 2021
From: CHEN, KE; DENG, ZHIPIN; CAI, XIAOXIA; WANG, CHEN; CHIU, YI-JEN; PENG, YA-TI; XU, LIDONG
To: INTEL CORPORATION
Reel/Frame 057767/0706 →
Continuity (3)
Continuation 16658803 · Oct 21, 2019
Continuation 15127785
Related Publication 20220020226A1 · Jan 20, 2022