IP Library › Granted Patent US 11,222,405
Granted Patent B2
US 11,222,405 · App. 16/816,930 · Granted Jan 11, 2022

Facial image enhancement method, device and electronic device

Inventor: Liang Zhang (Beijing, CN)
Assignee: Beijing Xiaomi Intelligent Technology Co., Ltd.
G06T5/002G06K9/00255G06T5/50G06T2207/20221G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,222,405
App. No.
16/816,930
Granted
Jan 11, 2022
Kind
B2
Abstract

A facial image enhancement method includes: determining a plurality of feature regions in a facial image according to feature points of the facial image; correcting an initial enhancement weight coefficient of each feature region of the feature regions and obtaining a weight map of discrete enhancement on the facial image; obtaining a weight map of continuous enhancement on the facial image according to the weight map of discrete enhancement on the facial image; and performing enhancing the facial image according to the weight map of continuous enhancement on the facial image to obtain an enhanced facial image.

Claims (62)

1. A facial image enhancement method, comprising:

determining a plurality of feature regions on a facial image according to feature points of the facial image;

correcting an initial enhancement weight coefficient of each feature region of the feature regions, and obtaining a weight map of discrete enhancement on the facial image;

obtaining a weight map of continuous enhancement on the facial image according to the weight map of discrete enhancement on the facial image; and

performing enhancing the facial image according to the weight map of continuous enhancement on the facial image to obtain an enhanced facial image,

wherein the correcting the initial enhancement weight coefficient of each feature region of the feature regions, and the obtaining the weight map of discrete enhancement on the facial image comprises:

obtaining the initial enhancement weight coefficient corresponding to each feature region of the feature regions;

determining a smoothing coefficient of each feature region of the feature regions and a physiological feature coefficient corresponding to the facial image; and

correcting the initial enhancement weight coefficient according to the smoothing coefficient of each feature region of the feature regions and the physiological feature coefficient, to obtain the weight map of discrete enhancement on the facial image.

2. The facial image enhancement method according to claim 1 , wherein the physiological feature coefficient comprises at least one of a gender coefficient or an age coefficient.

3. The facial image enhancement method according to claim 1 , wherein the determining the smoothing coefficient of each feature region of the feature regions comprises:

obtaining a variance and an average value for pixel values of a plurality of pixels in each feature region of the feature regions; and

determining the smoothing coefficient according to the variance and the average value.

4. The facial image enhancement method according to claim 1 , wherein the obtaining the weight map of continuous enhancement on the facial image according to the weight map of discrete enhancement on the facial image comprises:

determining the weight map of continuous enhancement on the facial image according to a guided image filtering algorithm, wherein an original grayscale image corresponding to the facial image is used as a guide map in the guided image filtering algorithm, and the weight map of discrete enhancement on the facial image is used as an input image in the guided image filtering algorithm.

5. The facial image enhancement method according to claim 1 , wherein the performing enhancing the facial image according to the weight map of continuous enhancement on the facial image to obtain an enhanced facial image comprises:

performing enhancing an original grayscale image corresponding to the facial image by an image enhancement algorithm, to obtain an enhanced grayscale image;

obtaining a fused grayscale image according to the weight map of continuous enhancement on the facial image, the original grayscale image, and the enhanced grayscale image; and

obtaining the enhanced facial image according to the fused grayscale image.

6. The facial image enhancement method according to claim 5 , wherein the performing enhancing the original grayscale image corresponding to the facial image by the image enhancement algorithm, to obtain the enhanced grayscale image comprises:

obtaining the enhanced grayscale image based on a high-contrast retention algorithm and the original grayscale image.

7. The facial image enhancement method according to claim 5 , wherein the obtaining the fused grayscale image according to the weight map of continuous enhancement on the facial image, the original grayscale image and the enhanced grayscale image comprises:

obtaining the fused grayscale image according to:

Y _ C (i,j) =M (i,j) *Y _ B (i,j) +(1− M (i,j) )* Y _ A (i,j) ;

wherein Y_C (i, j) is a fused grayscale image of a pixel located at (i, j), M (i, j) is an out-of-focus weight of the pixel located at (i, j) obtained according to the weight map of continuous enhancement on the facial image, Y_B (i, j) is an enhanced grayscale image of the pixel located at (i, j), and Y_A (i, j) is an original grayscale image of the pixel located at (i, j).

8. A facial image enhancement device, comprising:

a processor; and

a memory storing instructions executable by the processor;

wherein the processor is configured to:

determine a plurality of feature regions in a facial image according to feature points of the facial image;

correct an initial enhancement weight coefficient of each feature region of the feature regions and obtain a weight map of discrete enhancement on the facial image;

obtain a weight map of continuous enhancement on the facial image according to the weight map of discrete enhancement on the facial image; and

perform enhancing the facial image according to the weight map of continuous enhancement on the facial image to obtain an enhanced facial image,

wherein in correcting the initial enhancement weight coefficient of each feature region of the feature regions and obtaining the weight map of discrete enhancement on the facial image, the processor is further configured to:

obtain the initial enhancement weight coefficient corresponding to each feature region of the feature regions;

determine a smoothing coefficient of each feature region of the feature regions and a physiological feature coefficient corresponding to the facial image; and

correct the initial enhancement weight coefficient according to the smoothing coefficient of each feature region of the feature regions and the physiological feature coefficient to obtain the weight map of discrete enhancement on the facial image.

9. The facial image enhancement device according to claim 8 , wherein the physiological feature coefficient comprises at least one of a gender coefficient or an age coefficient.

10. The facial image enhancement device according to claim 8 , wherein the processor is further configured to:

obtain a variance and an average value for pixel values of a plurality of pixels in each feature region of the feature regions; and

determine the smoothing coefficient according to the variance and the average value.

11. The facial image enhancement device according to claim 8 , wherein the processor is further configured to:

determine the weight map of continuous enhancement on the facial image according to a guided image filtering algorithm, wherein an original grayscale image corresponding to the facial image is used as a guide map in the guided image filtering algorithm, and the weight map of discrete enhancement on the facial image is used as an input image in the guided image filtering algorithm.

12. The facial image enhancement device according to claim 8 , wherein the processor is further configured to:

perform enhancing an original grayscale image corresponding to the facial image by an image enhancement algorithm, to obtain an enhanced grayscale image;

obtain a fused grayscale image according to the weight map of continuous enhancement on the facial image, the original grayscale image, and the enhanced grayscale image; and

obtain the enhanced facial image according to the fused grayscale image.

13. The facial image enhancement device according to claim 12 , wherein the processor is further configured to:

obtain the enhanced grayscale image based on a high-contrast retention algorithm and the original grayscale image.

14. The facial image enhancement method according to claim 12 , wherein the processor is further configured to:

obtain the fused grayscale image according to:

Y _ C (i,j) =M (i,j) *Y _ B (i,j) +(1− M (i,j) )* Y _ A (i,j) ;

wherein, Y_C (i, j) is a fused grayscale image of a pixel located at (i, j), M (i, j) is an out-of-focus weight of the pixel located at (i, j) according to the weight map of continuous enhancement on the facial image, Y_B (i, j) is an enhanced grayscale image of the pixel located at (i, j), and Y_A (i, j) is an original grayscale image of the pixel located at (i, j).

15. A non-transitory computer-readable storage medium having stored thereon instructions that, when being executed by a processor of a device, cause the device to perform a facial image enhancement method, the method comprising:

determining a plurality of feature regions on a facial image according to feature points of the facial image;

correcting an initial enhancement weight coefficient of each feature region of the feature regions, and obtaining a weight map of discrete enhancement on the facial image;

obtaining a weight map of continuous enhancement on the facial image according to the weight map of discrete enhancement on the facial image; and

performing enhancing the facial image according to the weight map of continuous enhancement on the facial image to obtain an enhanced facial image,

wherein the correcting the initial enhancement weight coefficient of each feature region of the feature regions, and the obtaining the weight map of discrete enhancement on the facial image comprises:

obtaining the initial enhancement weight coefficient corresponding to each feature region of the feature regions;

determining a smoothing coefficient of each feature region of the feature regions and a physiological feature coefficient corresponding to the facial image; and

correcting the initial enhancement weight coefficient according to the smoothing coefficient of each feature region of the feature regions and the physiological feature coefficient, to obtain the weight map of discrete enhancement on the facial image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 12, 2020
From: ZHANG, LIANG
To: BEIJING XIAOMI INTELLIGENT TECHNOLOGY CO., LTD.
Reel/Frame 052099/0439 →
Priority Claims (1)
CN 201911019722.0 · Oct 24, 2019 · national
Continuity (1)
Related Publication 20210125311A1 · Apr 29, 2021