IP Library › Granted Patent US 10,733,715
Granted Patent B2
US 10,733,715 · App. 15/974,041 · Granted Aug 4, 2020

Image processing method and apparatus, electronic device and storage medium

Inventor: Chunjen Huang (Beijing, CN)
Assignee: BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO., LTD.
G06T5/50G06K9/00281G06T5/005G06T11/00G06T11/60G06K9/00228G06K9/00308G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,733,715
App. No.
15/974,041
Granted
Aug 4, 2020
Kind
B2
Abstract

Embodiments of the present disclosure provide an image processing method, an image processing apparatus, an electronic device and a storage medium, in order to improve a poor image effect when an image processing is performed for exchanging faces. The image processing method comprises: acquiring a first face image in an input image; extracting facial features of the first face image; determining a representation vector of the first face image according to the facial features; determining a target face image in a preset face image library according to the representation vector; and performing a face-exchange processing on the first face image according to the target face image, so as to obtain an output image.

Claims (97)

1. An image processing method, comprising:

acquiring a first face image in an input image;

extracting facial features of the first face image;

determining a representation vector of the first face image according to the facial features;

determining a target face image in a preset face image library according to the representation vector; and

performing a face-exchange processing on the first face image according to the target face image, so as to obtain an output image,

wherein the representation vector comprises N dimensions, where N is an integer greater than 1, and

wherein determining a representation vector of the first face image according to the facial features comprises:

estimating the facial features to obtain at least one representation probability value corresponding to each of the N dimensions, so as to obtain M representation probability values, where M is greater than or equal to N; and

performing a weighting processing on the M representation probability values, so as to obtain the representation vector.

2. The image processing method according to claim 1 , wherein acquiring the first face image in an input image comprises:

extracting a feature value of the input image;

performing an integration processing on the input image according to the feature value to obtain an integral image;

adopting an adaptive enhancement algorithm for distinguishing enhanced classifiers for faces and enhanced classifiers for non-faces in the integral image;

connecting the enhanced classifiers for faces to obtain a face image;

determining whether the face image is the first face image, if yes, acquiring the first face image.

3. The image processing method according to claim 2 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

4. The image processing method according to claim 3 , wherein the preprocessing at least comprises a face alignment; and

after the fourth face image is obtained, the image processing method further comprises:

performing a reverse face-alignment on the fourth face image, so as to obtain a sixth face image; and

rectifying the sixth face image by the loss function, so as to obtain the fifth face image.

5. The image processing method according to claim 1 , wherein estimating the facial features to obtain at least one representation probability value corresponding to each of the N dimensions comprises:

matching the facial features to a feature parameter of a target dimension, so as to obtain a plurality of matching values, in which the target dimension is one of the N dimensions of the representation vector; and

determining a matching value greater than or equal to a preset threshold in the plurality of matching values as a representation probability value corresponding to the target dimension.

6. The image processing method according to claim 5 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

7. The image processing method according to claim 6 , wherein the preprocessing at least comprises a face alignment; and

after the fourth face image is obtained, the image processing method further comprises:

performing a reverse face-alignment on the fourth face image, so as to obtain a sixth face image; and

rectifying the sixth face image by the loss function, so as to obtain the fifth face image.

8. The image processing method according to claim 1 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

9. The image processing method according to claim 8 , wherein the preprocessing at least comprises a face alignment; and

after the fourth face image is obtained, the image processing method further comprises:

performing a reverse face-alignment on the fourth face image, so as to obtain a sixth face image; and

rectifying the sixth face image by the loss function, so as to obtain the fifth face image.

10. The image processing method according to claim 1 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

11. The image processing method according to claim 10 , wherein the preprocessing at least comprises a face alignment; and

after the fourth face image is obtained, the image processing method further comprises:

performing a reverse face-alignment on the fourth face image, so as to obtain a sixth face image; and

rectifying the sixth face image by the loss function, so as to obtain the fifth face image.

12. An electronic device, comprising a housing, a processor, a memory, a circuit board, a power supply circuit, wherein, the circuit board is located in a space enclosed by the housing; the processor and the memory are positioned on the circuit board; the power supply circuit is configured to provide power for respective circuits or components of the electronic device; the memory is configured to store executable program codes; and the processor is configured to run a program corresponding to the executable program codes by reading the executable program codes stored in the memory, to perform the image processing method comprising:

acquiring a first face image in an input image;

extracting facial features of the first face image;

determining a representation vector of the first face image according to the facial features;

determining a target face image in a preset face image library according to the representation vector; and

performing a face-exchange processing on the first face image according to the target face image, so as to obtain an output image,

wherein the representation vector comprises N dimensions, where N is an integer greater than 1, and

wherein determining a representation vector of the first face image according to the facial features comprises:

estimating the facial features to obtain at least one representation probability value corresponding to each of the N dimensions, so as to obtain M representation probability values, where M is greater than or equal to N; and

performing a weighting processing on the M representation probability values, so as to obtain the representation vector.

13. The electronic device according to claim 12 , wherein estimating the facial features to obtain at least one representation probability value corresponding to each of the N dimensions comprises:

matching the facial features to a feature parameter of a target dimension, so as to obtain a plurality of matching values, in which the target dimension is one of the N dimensions of the representation vector; and

determining a matching value greater than or equal to a preset threshold in the plurality of matching values as a representation probability value corresponding to the target dimension.

14. The electronic device according to claim 13 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

15. The electronic device according to claim 12 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

16. The electronic device according to claim 15 , wherein the preprocessing at least comprises a face alignment; and

after the fourth face image is obtained, the image processing method further comprises:

performing a reverse face-alignment on the fourth face image, so as to obtain a sixth face image; and

rectifying the sixth face image by the loss function, so as to obtain the fifth face image.

17. The electronic device according to claim 12 , wherein performing a face-exchange processing on the first face image according to the target face image so as to obtain the output image comprises:

performing a preprocessing on a second face image in the target face image, so as to obtain a third face image;

replacing facial features of the third face image with the facial features of the first face image, so as to obtain a fourth face image;

rectifying the fourth face image by a loss function, so as to obtain a fifth face image; and

merging the fifth face image with an image other than the second face image in the target face image, so as to obtain the output image.

18. A non-transitory computer-readable storage medium having stored therein a computer program that, when executed by a processor of a device, causes the processor to perform the image processing method comprising:

acquiring a first face image in an input image;

extracting facial features of the first face image;

determining a representation vector of the first face image according to the facial features;

determining a target face image in a preset face image library according to the representation vector; and

performing a face-exchange processing on the first face image according to the target face image, so as to obtain an output image,

wherein the representation vector comprises N dimensions, where N is an integer greater than 1, and

wherein determining a representation vector of the first face image according to the facial features comprises:

estimating the facial features to obtain at least one representation probability value corresponding to each of the N dimensions, so as to obtain M representation probability values, where M is greater than or equal to N; and

performing a weighting processing on the M representation probability values, so as to obtain the representation vector.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2023
From: JOYINME PTE. LTD.
To: JUPITER PALACE PTE. LTD.
Reel/Frame 064158/0835 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2021
From: BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO., LTD.
To: JOYINME PTE. LTD.
Reel/Frame 055745/0877 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2018
From: HUANG, CHUNJEN
To: BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO., LTD.
Reel/Frame 045768/0878 →
Priority Claims (1)
CN 2017 1 0531744 · Jun 30, 2017 · national
Continuity (1)
Related Publication 20190005632A1 · Jan 3, 2019