IP Library › Granted Patent US 10,733,421
Granted Patent B2
US 10,733,421 · App. 15/913,314 · Granted Aug 4, 2020

Method for processing video, electronic device and storage medium

Inventor: Chiachi Huang (Beijing, CN)
Assignee: BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO., LTD.
G06K9/00248G06K9/00281G06K9/00711G06T7/11G06T11/00G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,733,421
App. No.
15/913,314
Granted
Aug 4, 2020
Kind
B2
Abstract

Embodiments of the present disclosure provide a method and a device for processing a video, an electronic device and a storage medium. The method includes: performing target recognition on each frame in an input video to obtain M frames containing a first face image, in which M is an integer greater than 1; replacing the first face image in the M frames with a target face image to obtain M first output frames; performing feature point locating on the first face image in a first frame in the M frames to obtain a first feature point set; extracting a target feature point set corresponding to a target region from the first feature point set; and performing image synthesis according to the target feature point set and the M first output frames to obtain M second output frames.

Claims (59)

1. A method for processing a video, comprising:

performing target recognition on each frame in an input video to obtain M frames containing a first face image, the M frames being obtained in time order, wherein M is an integer greater than 1;

replacing the first face image in the M frames with a target face image to obtain M first output frames;

performing feature point locating on the first face image in a first frame in the M frames to obtain a first feature point set;

performing feature point locating on the first face image in a third frame before the first frame in the M frames to obtain a third feature point set;

obtaining a positional difference between feature points of a first feature point subset and a third feature point subset, wherein the first feature point subset corresponds to a reference region of the first feature point set, and the third feature point subset corresponds to the reference region of the third feature point set;

when the difference is greater than a preset difference threshold, determining the region corresponding to the third feature point subset as a target region;

extracting a target feature point set corresponding to the target region from the first feature point set; and

performing image synthesis according to the target feature point set and the M first output frames to obtain M second output frames.

2. The method according to claim 1 , wherein performing the image synthesis according to the target feature point set and the M first output frames to obtain the M second output frames, comprises:

obtaining a face region image corresponding to the target feature point set; and

performing the image synthesis on the face region image and each first output frame to obtain the corresponding second output frame.

3. The method according to claim 2 , before performing the image synthesis according to the target feature point set and the M first output frames, further comprising:

performing feature point locating on the first face image in a second frame in the M frames to obtain a second feature point set; and

searching the target face image matching with the second feature point set from a preset face image base.

4. The method according to claim 1 , before performing the image synthesis according to the target feature point set and the M first output frames, further comprising:

performing feature point locating on the first face image in a second frame in the M frames to obtain a second feature point set; and

searching the target face image matching with the second feature point set from a preset face image base.

5. The method according to claim 1 , wherein replacing the first face image in the M frames to obtain the M first input frames comprises:

performing pre-processing on the target face image to obtain a target frame, wherein the pre-processing at least comprises face alignment processing, face image enhancement processing and normalization processing; and

replacing the first face image in the M frames with the target frame to obtain the M first output frames.

6. An electronic device, comprising: a housing, a processor, a memory, a circuit board and a power circuit, wherein, the circuit board is arranged inside a space enclosed by the housing, the processor and the memory are disposed on the circuit board; the power circuit is configured to provide power for individual circuits or components of the electronic device; the memory is configured to store executable program codes; and the processor, by reading the executable program codes stored in the memory, is configured to run programs corresponding to the executable program codes, so as to perform acts of:

performing target recognition on each frame in an input video to obtain M frames containing a first face image, the M frames being obtained in time order, wherein M is an integer greater than 1;

replacing the first face image in the M frames with a target face image to obtain M first output frames;

performing feature point locating on the first face image in a first frame in the M frames to obtain a first feature point set;

performing feature point locating on the first face image in a third frame before the first frame in the M frames to obtain a third feature point set obtaining a positional difference between feature points of a first feature point subset and a third feature point subset, wherein the first feature point subset corresponds to a reference region of the first feature point set, and the third feature point subset corresponds to the reference region of the third feature point set;

when the difference is greater than a preset difference threshold, determining the region corresponding to the third feature point subset as a target region;

extracting a target feature point set corresponding to the target region from the first feature point set; and

performing image synthesis according to the target feature point set and the M first output frames to obtain M second output frames.

7. The electronic device according to claim 6 , wherein the processor is configured to perform the image synthesis according to the target feature point set and the M first output frames to obtain the M second output frames, by acts of:

obtaining a face region image corresponding to the target feature point set; and

performing the image synthesis on the face region image and each first output frame to obtain the corresponding second output frame.

8. The electronic device according to claim 7 , wherein the processor is configured to perform acts of:

before performing the image synthesis according to the target feature point set and the M first output frames, performing feature point locating on the first face image in a second frame in the M frames to obtain a second feature point set; and

searching the target face image matching with the second feature point set from a preset face image base.

9. The electronic device according to claim 6 , wherein the processor is configured to perform acts of:

before performing the image synthesis according to the target feature point set and the M first output frames, performing feature point locating on the first face image in a second frame in the M frames to obtain a second feature point set; and

searching the target face image matching with the second feature point set from a preset face image base.

10. The electronic device according to claim 6 , wherein the processor is configured to replace the first face image in the M frames to obtain the M first input frames by acts of:

performing pre-processing on the target face image to obtain a target frame, wherein the pre-processing at least comprises face alignment processing, face image enhancement processing and normalization processing; and

replacing the first face image in the M frames with the target frame to obtain the M first output frames.

11. A non-transitory computer-readable storage medium, having stored therein instructions that, when a processor executes the instructions, cause the processor to perform a method for processing a video, the method comprising:

performing target recognition on each frame in an input video to obtain M frames containing a first face image, the M frames being obtained in time order, wherein M is an integer greater than 1;

replacing the first face image in the M frames with a target face image to obtain M first output frames;

performing feature point locating on the first face image in a first frame in the M frames to obtain a first feature point set;

performing feature point locating on the first face image in a third frame before the first frame in the M frames to obtain a third feature point set;

obtaining a positional difference between feature points of a first feature point subset and a third feature point subset, wherein the first feature point subset corresponds to a reference region of the first feature point set, and the third feature point subset corresponds to the reference region of the third feature point set;

when the difference is greater than a preset difference threshold, determining the region corresponding to the third feature point subset as a target region;

extracting a target feature point set corresponding to the target region from the first feature point set; and

performing image synthesis according to the target feature point set and the M first output frames to obtain M second output frames.

12. The non-transitory computer-readable storage medium according to claim 11 , wherein performing the image synthesis according to the target feature point set and the M first output frames to obtain the M second output frames, comprises:

obtaining a face region image corresponding to the target feature point set; and

performing the image synthesis on the face region image and each first output frame to obtain the corresponding second output frame.

13. The non-transitory computer-readable storage medium according to claim 11 , wherein the method further comprises:

before performing the image synthesis according to the target feature point set and the M first output frames, performing feature point locating on the first face image in a second frame in the M frames to obtain a second feature point set; and

searching the target face image matching with the second feature point set from a preset face image base.

14. The non-transitory computer-readable storage medium according to claim 11 , wherein replacing the first face image in the M frames to obtain the M first input frames comprises:

performing pre-processing on the target face image to obtain a target frame, wherein the pre-processing at least comprises face alignment processing, face image enhancement processing and normalization processing; and

replacing the first face image in the M frames with the target frame to obtain the M first output frames.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2023
From: JOYINME PTE. LTD.
To: JUPITER PALACE PTE. LTD.
Reel/Frame 064190/0756 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 12, 2021
From: BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO., LTD.
To: JOYINME PTE. LTD.
Reel/Frame 055658/0541 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 6, 2018
From: HUANG, CHIACHI
To: BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO., LTD.
Reel/Frame 045122/0464 →
Priority Claims (1)
CN 2017 1 0531790 · Jun 30, 2017 · national
Continuity (1)
Related Publication 20190005305A1 · Jan 3, 2019