Granted Patent B2

US 12,293,465 · App. 17/751,496 · Granted May 6, 2025

Inventors: Xiangkai Lin (Shenzhen, CN); Sheng Wang (Shenzhen, CN)

Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

G06T17/205G06T3/40G06T7/64G06T7/70G06T15/04G06V40/165G06T2200/04G06T2200/08G06T2207/10028G06T2207/20216G06T2207/20221G06T2207/30201G06T2210/56

Abstract

Claims (97)

1. A method for constructing a three-dimensional (3D) facial mesh, performed by a computing device, the method comprising:

obtaining a facial point cloud of a target object;

generating a facial texture image of the target object from the facial point cloud and n face images corresponding to n head postures of the target object, n being a positive number;

performing grid downsampling on the facial point cloud to obtain a downsampled facial point cloud;

determining, through an expansion calculation, pixel coordinates on the facial texture image of the target object that correspond to 3D data points in the downsampled facial point cloud as index information of the 3D data points;

performing triangulation on pixels on the facial texture image to obtain triangulation information, the triangulation information being used for indicating triangular topologies corresponding to the pixels;

constructing an initial 3D facial mesh according to the triangulation information, the index information and the downsampled facial point cloud;

determining a non-core region in the initial 3D facial mesh, the non-core region being a non-facial region in the initial 3D facial mesh;

smoothing the non-core region in the initial 3D facial mesh to obtain a smoothed non-core region; and

replacing the non-core region in the initial 3D facial mesh with the smoothed non-core region to obtain a 3D facial mesh of the target object.

2. The method according to claim 1 , wherein the determining the non-core region in the initial 3D facial mesh comprises:

determining non-facial key pixels in the facial texture image;

determining non-facial key 3D data points and triangular topologies corresponding to the non-facial key 3D data points based on the non-facial key pixels, the triangulation information and the index information; and

determining a region formed by the triangular topologies corresponding to the non-facial key 3D data points as the non-core region.

3. The method according to claim 2 , wherein the determining the non-facial key pixels in the facial texture image comprises:

determining a convex hull corresponding to a core region in the facial texture image, the core region being a region covered by facial key pixels in the facial texture image;

enlarging the convex hull based on a dilation threshold to obtain a target region; and

determining pixels in a region other than the target region in the facial texture image as the non-facial key pixels.

4. The method according to claim 1 , wherein the performing grid downsampling on the facial point cloud to obtain the downsampled facial point cloud comprises:

defining a plurality of grids corresponding to the non-core region in a 3D space according to a first grid density, and defining a plurality of grids corresponding to the core region according to a second grid density;

fusing the facial point cloud into a plurality of grids in the 3D space; and

averaging, under a same coordinate dimension, 3D coordinates of 3D data points in the facial point cloud included in each grid to obtain the downsampled facial point cloud.

5. The method according to claim 1 , wherein the determining, through the expansion calculation, the pixel coordinates on the facial texture image of the target object that correspond to 3D data points in the downsampled facial point cloud as the index information of the 3D data points comprises:

obtaining 3D coordinates of the 3D data points in the downsampled facial point cloud in a 3D coordinate system; and

determining, according to the 3D coordinates of the 3D data points through a cylindrical expansion calculation, the pixel coordinates on the facial texture image that correspond to the 3D data points, as the index information.

6. The method according to claim 1 , wherein the generation of the facial texture image further comprises:

obtaining a mapping relationship between the 3D data points in the facial point cloud and pixels on each of the n face images;

respectively obtaining image regions corresponding to the head postures from the n face images to obtain n valid regions;

for an i th valid region in the n valid regions, obtaining target 3D data points corresponding to pixels in the i th valid region according to the mapping relationship, i being a positive integer less than or equal to n;

obtaining target pixels, which are on a regional texture image corresponding to the i th valid region, corresponding to the target 3D data points according to the index information;

rendering the target pixels to generate the regional texture image corresponding to the i th valid region, pixel values of the target pixels being determined according to pixel values of the pixels in the i th valid region; and

performing image fusion on the n regional texture images to generate the facial texture image of the target object.

7. The method according to claim 6 , wherein the respectively obtaining image regions corresponding to the head postures from the n face images to obtain the n valid regions comprises:

for the i th face image in the n face images, detecting facial key points in the i th face image; and

obtaining a valid region in the i th face image according to positions of the facial key points and a predefined rule.

8. The method according to claim 6 , after the performing image fusion on the n regional texture images to generate the facial texture image of the target object, the method further comprising:

obtaining according to the index information, pixel values of pixels, which correspond to the 3D data points, on the facial texture image; and

rendering the 3D facial mesh according to the pixel values of the 3D data points, to generate a textured 3D facial mesh.

9. A computing device, comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement a method for constructing a three-dimensional (3D) facial mesh, the method comprising:

obtaining a facial point cloud of a target object;

generating a facial texture image of the target object from the facial point cloud and n face images corresponding to n head postures of the target object, n being a positive number;

performing grid downsampling on the facial point cloud to obtain a downsampled facial point cloud;

determining, through an expansion calculation, pixel coordinates on the facial texture image of the target object that correspond to 3D data points in the downsampled facial point cloud as index information of the 3D data points;

performing triangulation on pixels on the facial texture image to obtain triangulation information, the triangulation information being used for indicating triangular topologies corresponding to the pixels;

constructing an initial 3D facial mesh according to the triangulation information, the index information and the downsampled facial point cloud;

determining a non-core region in the initial 3D facial mesh, the non-core region being a non-facial region in the initial 3D facial mesh;

smoothing the non-core region in the initial 3D facial mesh to obtain a smoothed non-core region; and

replacing the non-core region in the initial 3D facial mesh with the smoothed non-core region to obtain a 3D facial mesh of the target object.

10. The computing device according to claim 9 , wherein the determining the non-core region in the initial 3D facial mesh comprises:

determining non-facial key pixels in the facial texture image;

determining non-facial key 3D data points and triangular topologies corresponding to the non-facial key 3D data points based on the non-facial key pixels, the triangulation information and the index information; and

determining a region formed by the triangular topologies corresponding to the non-facial key 3D data points as the non-core region.

11. The computing device according to claim 10 , wherein the determining the non-facial key pixels in the facial texture image comprises:

determining a convex hull corresponding to a core region in the facial texture image, the core region being a region covered by facial key pixels in the facial texture image;

enlarging the convex hull based on a dilation threshold to obtain a target region; and

determining pixels in a region other than the target region in the facial texture image as the non-facial key pixels.

12. The computing device according to claim 9 , wherein the performing grid downsampling on the facial point cloud to obtain the downsampled facial point cloud comprises:

defining a plurality of grids corresponding to the non-core region in a 3D space according to a first grid density, and defining a plurality of grids corresponding to the core region according to a second grid density;

fusing the facial point cloud into a plurality of grids in the 3D space; and

averaging, under a same coordinate dimension, 3D coordinates of 3D data points in the facial point cloud included in each grid to obtain the downsampled facial point cloud.

13. The computing device according to claim 9 , wherein the determining, through the expansion calculation, the pixel coordinates on the facial texture image of the target object that correspond to 3D data points in the downsampled facial point cloud as the index information of the 3D data points comprises:

obtaining 3D coordinates of the 3D data points in the downsampled facial point cloud in a 3D coordinate system; and

determining, according to the 3D coordinates of the 3D data points through a cylindrical expansion calculation, the pixel coordinates on the facial texture image that correspond to the 3D data points, as the index information.

14. The computing device according to claim 9 , wherein the generation of the facial texture image further comprises:

obtaining a mapping relationship between the 3D data points in the facial point cloud and pixels on each of the n face images;

respectively obtaining image regions corresponding to the head postures from the n face images to obtain n valid regions;

for an i th valid region in the n valid regions, obtaining target 3D data points corresponding to pixels in the i th valid region according to the mapping relationship, i being a positive integer less than or equal to n;

obtaining target pixels, which are on a regional texture image corresponding to the i th valid region, corresponding to the target 3D data points according to the index information;

rendering the target pixels to generate the regional texture image corresponding to the i th valid region, pixel values of the target pixels being determined according to pixel values of the pixels in the i th valid region; and

performing image fusion on the n regional texture images to generate the facial texture image of the target object.

15. The computing device according to claim 14 , wherein the respectively obtaining image regions corresponding to the head postures from the n face images to obtain the n valid regions comprises:

for the i th face image in the n face images, detecting facial key points in the i th face image; and

obtaining a valid region in the i th face image according to positions of the facial key points and a predefined rule.

16. The computing device according to claim 14 , wherein after the performing image fusion on the n regional texture images to generate the facial texture image of the target object, the method further comprises:

obtaining according to the index information, pixel values of pixels, which correspond to the 3D data points, on the facial texture image; and

rendering the 3D facial mesh according to the pixel values of the 3D data points, to generate a textured 3D facial mesh.

17. A non-transitory computer-readable storage medium, storing at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement a method for constructing a three-dimensional (3D) facial mesh, the method comprising:

obtaining a facial point cloud of a target object;

generating a facial texture image of the target object from the facial point cloud and n face images corresponding to n head postures of the target object, n being a positive number;

performing grid downsampling on the facial point cloud to obtain a downsampled facial point cloud;

determining, through an expansion calculation, pixel coordinates on the facial texture image of the target object that correspond to 3D data points in the downsampled facial point cloud as index information of the 3D data points;

performing triangulation on pixels on the facial texture image to obtain triangulation information, the triangulation information being used for indicating triangular topologies corresponding to the pixels;

constructing an initial 3D facial mesh according to the triangulation information, the index information and the downsampled facial point cloud;

determining a non-core region in the initial 3D facial mesh, the non-core region being a non-facial region in the initial 3D facial mesh;

smoothing the non-core region in the initial 3D facial mesh to obtain a smoothed non-core region; and

replacing the non-core region in the initial 3D facial mesh with the smoothed non-core region to obtain a 3D facial mesh of the target object.

18. The non-transitory computer-readable storage medium according to claim 17 , wherein the determining the non-core region in the initial 3D facial mesh comprises:

determining non-facial key pixels in the facial texture image;

determining non-facial key 3D data points and triangular topologies corresponding to the non-facial key 3D data points based on the non-facial key pixels, the triangulation information and the index information; and

determining a region formed by the triangular topologies corresponding to the non-facial key 3D data points as the non-core region.

19. The non-transitory computer-readable storage medium according to claim 17 , wherein the performing grid downsampling on the facial point cloud to obtain the downsampled facial point cloud comprises:

defining a plurality of grids corresponding to the non-core region in a 3D space according to a first grid density, and defining a plurality of grids corresponding to the core region according to a second grid density;

fusing the facial point cloud into a plurality of grids in the 3D space; and

averaging, under a same coordinate dimension, 3D coordinates of 3D data points in the facial point cloud included in each grid to obtain the downsampled facial point cloud.

20. The non-transitory computer-readable storage medium according to claim 17 , wherein the determining, through the expansion calculation, the pixel coordinates on the facial texture image of the target object that correspond to 3D data points in the downsampled facial point cloud as the index information of the 3D data points comprises:

obtaining 3D coordinates of the 3D data points in the downsampled facial point cloud in a 3D coordinate system; and

determining, according to the 3D coordinates of the 3D data points through a cylindrical expansion calculation, the pixel coordinates on the facial texture image that correspond to the 3D data points, as the index information.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (22)