IP Library Granted Patent US 11,037,281
Granted Patent B2
US 11,037,281 · App. 16/859,331 · Granted Jun 15, 2021

Image fusion method and device, storage medium and terminal

Inventors: Keyi Shen (Shenzhen, CN); Pei Cheng (Shenzhen, CN); Mengren Qian (Shenzhen, CN); Bin Fu (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06T5/50G06K9/00248G06T7/55G06T15/50G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,037,281
App. No.
16/859,331
Granted
Jun 15, 2021
Kind
B2
Abstract

Embodiments of this application disclose an image fusion method performed by a computing device. The method includes the following steps: obtaining source face image data of a current to-be-fused image and resource configuration information of a current to-be-fused resource, performing image recognition processing on the source face image data, to obtain source face feature points corresponding to the source face image data, and generating a source face three-dimensional grid of the source face image data according to the source face feature points, performing grid fusion by using a resource face three-dimensional grid and the source face three-dimensional grid to generate a target face three-dimensional grid, and performing face complexion fusion by using source complexion data of the source face image data and resource complexion data of resource face image data on the target face three-dimensional grid, to generate fused target face image data.

Claims (71)

1. An image fusion method performed at a computing device having a processor and memory and a plurality of programs stored in the memory, the method comprising:

obtaining source face image data of a current to-be-fused image and resource configuration information of a current to-be-fused resource, the resource configuration information comprising resource face image data, resource complexion data, and a resource face three-dimensional grid;

obtaining source face feature points from the source face image data through image recognition;

generating a source face three-dimensional grid of the source face image data according to the source face feature points;

performing grid fusion to the resource face three-dimensional grid and the source face three-dimensional grid to generate a target face three-dimensional grid; and

performing face complexion fusion on the target face three-dimensional grid by using source complexion data of the source face image data and the resource complexion data of the resource face image data, to generate fused target face image data.

2. The method according to claim 1 , wherein the obtaining source face feature points from the source face image data through image recognition comprises:

obtaining reference feature points of the source face image data through the image recognition;

extracting three-dimensional depth information from the reference feature points, to obtain the source face feature points corresponding to the reference feature points; and

generating the source face three-dimensional grid according to the source face feature points.

3. The method according to claim 1 , wherein the performing grid fusion to the resource face three-dimensional grid and the source face three-dimensional grid to generate a target face three-dimensional grid comprises:

performing grid supplementing on the source face three-dimensional grid according to a symmetry of the source face image data after detecting that types of the source face three-dimensional grid and the resource face three-dimensional grid are inconsistent;

generating a candidate face three-dimensional grid whose type is consistent with that of the resource face three-dimensional grid; and

performing the grid fusion to the candidate face three-dimensional grid and the resource face three-dimensional grid to generate the target face three-dimensional grid.

4. The method according to claim 3 , wherein the performing face complexion fusion on the target face three-dimensional grid by using source complexion data of the source face image data and resource complexion data of the resource image data, to generate fused target face image data comprises:

performing complexion balance on the source face image data, to obtain average complexion data of the source face image data;

performing complexion filling on the candidate face three-dimensional grid based on the source complexion data of the source face image data and the average complexion data, to generate candidate face image data; and

performing face complexion fusion on the target face three-dimensional grid using candidate complexion data of the candidate face image data and the resource complexion data of the resource face image data, to generate the fused target face image data, wherein the candidate complexion data comprises the source complexion data and the average complexion data.

5. The method according to claim 4 , wherein the performing face complexion fusion on the target face three-dimensional grid using candidate complexion data of the candidate face image data and the resource complexion data of the resource face image data, to generate the fused target face image data comprises:

obtaining candidate pixel points in the candidate complexion data and resource pixel points in the resource complexion data;

calculating target pixel points based on the candidate pixel points and the resource pixel points using a fusion degree; and

generating the target face image data according to the target pixel points.

6. The method according to claim 1 , further comprising:

obtaining a light source type corresponding to the source face image data according to the source complexion data of the source face image data; and

adding a light effect on the target face image data corresponding to the light source type.

7. The method according to claim 1 , further comprising:

adjusting a current display position of the target face image data based on coordinate information indicated by the resource face image data.

8. The method according to claim 1 , further comprising:

in accordance with a determination that a first display region of the target face image data is smaller than a second display region of the source face image data, filling face edge on a part of the second display region except the first display region.

9. A computing device, comprising: a processor and memory, the memory storing a plurality of computer programs, wherein the computer programs, when executed by the processor, perform operations including:

obtaining source face image data of a current to-be-fused image and resource configuration information of a current to-be-fused resource, the resource configuration information comprising resource face image data, resource complexion data, and a resource face three-dimensional grid;

obtaining source face feature points from the source face image data through image recognition;

generating a source face three-dimensional grid of the source face image data according to the source face feature points;

performing grid fusion to the resource face three-dimensional grid and the source face three-dimensional grid to generate a target face three-dimensional grid; and

performing face complexion fusion on the target face three-dimensional grid by using source complexion data of the source face image data and the resource complexion data of the resource face image data, to generate fused target face image data.

10. The computing device according to claim 9 , wherein the obtaining source face feature points from the source face image data through image recognition comprises:

obtaining reference feature points of the source face image data through the image recognition;

extracting three-dimensional depth information from the reference feature points, to obtain the source face feature points corresponding to the reference feature points; and

generating the source face three-dimensional grid according to the source face feature points.

11. The computing device according to claim 9 , wherein the performing grid fusion to the resource face three-dimensional grid and the source face three-dimensional grid to generate a target face three-dimensional grid comprises:

performing grid supplementing on the source face three-dimensional grid according to a symmetry of the source face image data after detecting that types of the source face three-dimensional grid and the resource face three-dimensional grid are inconsistent;

generating a candidate face three-dimensional grid whose type is consistent with that of the resource face three-dimensional grid; and

performing the grid fusion to the candidate face three-dimensional grid and the resource face three-dimensional grid to generate the target face three-dimensional grid.

12. The computing device according to claim 11 , wherein the performing face complexion fusion on the target face three-dimensional grid by using source complexion data of the source face image data and resource complexion data of the resource image data, to generate fused target face image data comprises:

performing complexion balance on the source face image data, to obtain average complexion data of the source face image data;

performing complexion filling on the candidate face three-dimensional grid based on the source complexion data of the source face image data and the average complexion data, to generate candidate face image data; and

performing face complexion fusion on the target face three-dimensional grid using candidate complexion data of the candidate face image data and the resource complexion data of the resource face image data, to generate the fused target face image data, wherein the candidate complexion data comprises the source complexion data and the average complexion data.

13. The computing device according to claim 12 , wherein the performing face complexion fusion on the target face three-dimensional grid using candidate complexion data of the candidate face image data and the resource complexion data of the resource face image data, to generate the fused target face image data comprises:

obtaining candidate pixel points in the candidate complexion data and resource pixel points in the resource complexion data;

calculating target pixel points based on the candidate pixel points and the resource pixel points using a fusion degree; and

generating the target face image data according to the target pixel points.

14. The computing device according to claim 9 , wherein the operations further comprise:

obtaining a light source type corresponding to the source face image data according to the source complexion data of the source face image data; and

adding a light effect on the target face image data corresponding to the light source type.

15. The computing device according to claim 9 , wherein the operations further comprise:

adjusting a current display position of the target face image data based on coordinate information indicated by the resource face image data.

16. The computing device according to claim 9 , wherein the operations further comprise:

in accordance with a determination that a first display region of the target face image data is smaller than a second display region of the source face image data, filling face edge on a part of the second display region except the first display region.

17. A non-transitory computer storage medium, storing a plurality of instructions, the instructions being configured for, when executed by a processor a computing device, perform operations including:

obtaining source face image data of a current to-be-fused image and resource configuration information of a current to-be-fused resource, the resource configuration information comprising resource face image data, resource complexion data, and a resource face three-dimensional grid;

obtaining source face feature points from the source face image data through image recognition;

generating a source face three-dimensional grid of the source face image data according to the source face feature points;

performing grid fusion to the resource face three-dimensional grid and the source face three-dimensional grid to generate a target face three-dimensional grid; and

performing face complexion fusion on the target face three-dimensional grid by using source complexion data of the source face image data and the resource complexion data of the resource face image data, to generate fused target face image data.

18. The non-transitory computer storage medium according to claim 17 , wherein the operations further comprise:

obtaining a light source type corresponding to the source face image data according to the source complexion data of the source face image data; and

adding a light effect on the target face image data corresponding to the light source type.

19. The non-transitory computer storage medium according to claim 17 , wherein the operations further comprise:

adjusting a current display position of the target face image data based on coordinate information indicated by the resource face image data.

20. The non-transitory computer storage medium according to claim 17 , wherein the operations further comprise:

in accordance with a determination that a first display region of the target face image data is smaller than a second display region of the source face image data, filling face edge on a part of the second display region except the first display region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2020
From: SHEN, KEYI; CHENG, PEI; QIANG, MENGREN; FU, BIN
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED35
Reel/Frame 052800/0478 →
Priority Claims (1)
CN 201711173149.X · Nov 22, 2017 · national
Continuity (2)
Continuation PCTCN2018116832 · Nov 22, 2018
Related Publication 20200258206A1 · Aug 13, 2020
Cited By (1)
US 12,586,285