IP Library Granted Patent US 10,096,089
Granted Patent B2
US 10,096,089 · App. 15/398,503 · Granted Oct 9, 2018

Accelerated skin smoothing effect

Inventor: Andrei Igorevich Kopysov (Minsk, RU)
Assignee: Facebook, Inc.
G06T5/002G06T5/50G06T7/13H04N1/628G06T2207/10024G06T2207/20024G06T2207/20221G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,096,089
App. No.
15/398,503
Granted
Oct 9, 2018
Kind
B2
Abstract

In one embodiment, a method includes accessing an original image of a person's face to be smoothed. A second image may be generated to include color information corresponding to those of the original image. The color information of the second image may be blurred. A filter image may be generated and may include an Alpha channel and color channels. The Alpha channel may include a mask representing opaque and transparent information, and the color channels may include a smoothed original image. High frequency color information may be determined from the color information of the original image and used to modify the color information of the filter image. A display may be outputted based on an overlay of the modified filter image over the original image.

Claims (62)

1. A method, comprising:

accessing, using a processing system, a first digital image of an object, the first digital image comprising color information;

generating, using the processing system, a second digital image comprising color channels, wherein the color channels comprise color information corresponding to the color information of the first digital image;

for each of the color channels of the second digital image, blurring, using the processing system, the information comprised in the channel;

generating, using the processing system, a third digital image comprising an Alpha channel and color channels, wherein the Alpha channel comprises a mask representing opaque information and transparent information, and wherein the color channels of the third digital image comprise color information determined from a smoothing of the color information of the first digital image;

determining, using the processing system, high frequency color information from the color information of the first digital image;

modifying, using the processing system, the color information of the third digital image using the high frequency color information; and

outputting, using the processing system, a display based on an overlay of the modified third digital image over the first digital image.

2. The method of claim 1 , wherein the second digital image further comprises a first channel comprising brightness information associated with the first digital image, the method further comprising:

blurring, using the processing system, the brightness information comprised in the first channel of the second digital image;

detecting, using the processing system, edge information associated with the object based on the blurred brightness information of the second digital image; and

modifying, using the processing system, the blurred color information of the second digital image based on the edge information associated with the object;

wherein the smoothing of the color information of the first digital image is based on the edge information associated with the object and the modified blurred color information of the second digital image.

3. The method of claim 2 , wherein the first channel of the second digital image is an Alpha channel.

4. The method of claim 2 , wherein the brightness information comprised in the first channel of the second digital image is computed by squaring a brightness information of the first digital image.

5. The method of claim 2 , wherein the detecting of the edge information is further based on the blurred color information of the second digital image.

6. The method of claim 2 , wherein the detecting of the edge information comprises calculating brightness variance coefficients.

7. The method of claim 6 , wherein the modifying of the blurred color information of the second digital image comprises:

applying weights to the blurred color information of the second digital image, wherein the weights are based on the brightness variance coefficients.

8. The method of claim 6 , further comprising:

storing the brightness variance coefficients in the first channel of the second digital image; and

blurring the brightness variance coefficients in the first channel and the modified blurred color information in the color channels of the second digital image.

9. The method of claim 2 , wherein the modified blurred color information of the second digital image comprises edge information represented in dark pixels.

10. The method of claim 1 , wherein the second digital image's color channels comprises a red channel for containing red color information, a green channel for containing green color information, and a blue channel for containing blue color information.

11. The method of claim 1 , wherein the blurring is an average blurring based on a predetermined kernel radius.

12. The method of claim 1 , wherein the blurring is processed using a Graphical Processing Unit of the processing system.

13. The method of claim 12 , wherein the blurring comprises separately performed horizontal blurring and vertical blurring.

14. The method of claim 1 , wherein the determining of the high frequency color information is based on a second predetermined kernel size.

15. The method of claim 1 ,

wherein the transparent information of the mask defines a first region substantially corresponding to eyes, mouth, and hair; and

wherein the opaque information of the mask defines a second region substantially corresponding to facial skin.

16. The method of claim 1 ,

wherein the first digital image comprises a face with facial skin and an eye; and

wherein the display comprises an image comprising a modification of the face with the facial skin being smoothed and the eye not being smoothed.

17. A system, comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:

access a first digital image of an object, the first digital image comprising color information;

generate a second digital image comprising color channels, wherein the color channels comprise color information corresponding to the color information of the first digital image;

for each of the color channels of the second digital image, blur the information comprised in the channel;

generate a third digital image comprising an Alpha channel and color channels, wherein the Alpha channel comprises a mask representing opaque information and transparent information, and wherein the color channels of the third digital image comprise color information determined from a smoothing of the color information of the first digital image;

determine high frequency color information from the color information of the first digital image;

modify the color information of the third digital image using the high frequency color information; and

output a display based on an overlay of the modified third digital image over the first digital image.

18. The system of claim 17 , wherein the second digital image further comprises a first channel comprising brightness information associated with the first digital image, wherein the processors are further operable when executing the instructions to:

blur the brightness information comprised in the first channel of the second digital image;

detect edge information associated with the object based on the blurred brightness information of the second digital image; and

modify the blurred color information of the second digital image based on the edge information associated with the object;

wherein the smoothing of the color information of the first digital image is based on the edge information associated with the object and the modified blurred color information of the second digital image.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

access a first digital image of an object, the first digital image comprising color information;

generate a second digital image comprising color channels, wherein the color channels comprise color information corresponding to the color information of the first digital image;

for each of the color channels of the second digital image, blur the information comprised in the channel;

generate a third digital image comprising an Alpha channel and color channels, wherein the Alpha channel comprises a mask representing opaque information and transparent information, and wherein the color channels of the third digital image comprise color information determined from a smoothing of the color information of the first digital image;

determine high frequency color information from the color information of the first digital image;

modify the color information of the third digital image using the high frequency color information; and

output a display based on an overlay of the modified third digital image over the first digital image.

20. The media of claim 19 , wherein the second digital image further comprises a first channel comprising brightness information associated with the first digital image, wherein the software is further operable when executed to:

blur the brightness information comprised in the first channel of the second digital image;

detect edge information associated with the object based on the blurred brightness information of the second digital image; and

modify the blurred color information of the second digital image based on the edge information associated with the object;

wherein the smoothing of the color information of the first digital image is based on the edge information associated with the object and the modified blurred color information of the second digital image.

Assignments (3)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE MIDDLE AND LAST NAME OF INVENTOR PREVIOUSLY RECORDED ON REEL 041857 FRAME 0065. ASSIGNOR(S) HEREBY CONFIRMS THE MIDDLE NAME AND LAST NAME TO BE CORRECTED TO IGOREVICH KOPYSOV. Recorded Sep 27, 2018
From: KOPYSOV, ANDREI IGOREVICH
To: FACEBOOK, INC.
Reel/Frame 047635/0138 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2017
From: IGOREVICH, ANDREI KOPYSOV
To: FACEBOOK, INC.
Reel/Frame 041857/0065 →
Continuity (1)
Related Publication 20180189932A1 · Jul 5, 2018