Granted Patent B2

US 12,192,586 · App. 17/708,876 · Granted Jan 7, 2025

Inventors: Wenkang Xu (Shenzhen, CN); Wei Huang (Munich, DE); Junhao Zhang (Shanghai, CN); Xiaolin Huang (Shenzhen, CN)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

H04N21/4788G06T7/40G06T19/20G06V10/143G06V10/56G06V10/82G06V40/166G06V40/168G06V40/174H04N7/157G06T2207/10024G06T2207/10028G06T2207/30201G06T2219/2016G06V20/58G06V20/59

Abstract

Claims (74)

1. A method comprising:

obtaining, based on a face image of a user in a video call process, a key feature point of a facial expression of the user, wherein the face image comprises N infrared images of the user and a color face image of the user, and wherein N is an integer greater than 0;

obtaining, based on the N infrared images and the color face image, a three-dimensional (3D) head point cloud information of the user;

constructing, based on the 3D head point cloud information, a 3D head image of the user;

driving, using the key feature point, the 3D head image to obtain a first target 3D avatar of the user, wherein the first target 3D avatar has an expression of the user;

rotating, based on a preset presentation angle, the first target 3D avatar to obtain a second target 3D avatar at the preset presentation angle; and

sending, to a peer video call device, the second target 3D avatar.

2. The method of claim 1 , further comprising:

obtaining, based on the color face image, a face texture feature of the user; and

further constructing, based on the face texture feature, the 3D head image that is a color image, wherein the 3D head point cloud information comprises first 3D head point cloud information of the user or second 3D head point cloud information of the user.

3. The method of claim 1 , further comprising:

inputting the color face image and the N infrared images into a feature extraction model to obtain 3D head point cloud information of the user and a face texture feature of the user; and

further constructing, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

4. The method of claim 3 , wherein the feature extraction model comprises a 3D head feature extraction network and a texture feature extraction network, and wherein the method further comprises:

further inputting the color face image and the N infrared images into the 3D head feature extraction network to obtain the 3D head point cloud information; and

inputting the color face image into the texture feature extraction network to obtain the face texture feature.

5. An apparatus comprising:

a non-transitory memory configured to store instructions; and

a processor coupled to the non-transitory memory, wherein when executed by the processor, the instructions cause the apparatus to:

obtain, based on a face image of a user in a video call process, a key feature point of a facial expression of the user, wherein the face image comprises N infrared images of the user and a color face image of the user, and wherein N is an integer greater than 0;

obtain, based on the N infrared images and the color face image, a three-dimensional (3D) head point cloud information of the user;

construct, based on the 3D head point cloud information, a 3D head image of the user;

drive, using the key feature point, the 3D head image to obtain a first target 3D avatar of the user, wherein the first target 3D avatar has an expression of the user;

rotate, based on a preset presentation angle, the first target 3D avatar to obtain a second target 3D avatar at the preset presentation angle; and

send, to a peer video call device, the second target 3D avatar.

6. The apparatus of claim 5 , wherein when executed by the processor, the instructions further cause the apparatus to:

obtain, based on the color face image, a face texture feature of the user; and

further construct, based on the face texture feature, the 3D head image that is a color image, wherein the 3D head point cloud information comprises first 3D head point cloud information of the user or second 3D head point cloud information of the user.

7. The apparatus of claim 5 , wherein when executed by the processor, the instructions further cause the apparatus to:

input the color face image and the N infrared images into a feature extraction model to obtain 3D head point cloud information of the user and a face texture feature of the user; and

further construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

8. The apparatus of claim 7 , wherein the feature extraction model comprises a 3D head feature extraction network and a texture feature extraction network, and wherein when executed by the processor, the instructions further cause the apparatus to:

input the color face image and the N infrared images into the 3D head feature extraction network to obtain the 3D head point cloud information; and

input the color face image into the texture feature extraction network to obtain the face texture feature.

9. The apparatus of claim 8 , wherein the 3D head feature extraction network is a neural network that uses an encoder-decoder architecture, and wherein when executed by the processor, the instructions further cause the apparatus to:

obtain, based on the color face image and the N infrared images, N image pairs, wherein each of the N image pairs comprises the color face image and an infrared image of the user from the N infrared images; and

input the N image pairs into the neural network to obtain the 3D head point cloud information.

10. The apparatus of claim 9 , wherein when executed by the processor, the instructions further cause the apparatus to obtain, based on the N infrared images, the preset presentation angle.

11. The apparatus of claim 5 , wherein the face image is a color depth image, and wherein when executed by the processor, the instructions further cause the apparatus to:

obtain, based on the color depth image, 3D head point cloud information of the user and a face texture feature of the user; and

construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

12. The apparatus of claim 11 , wherein when executed by the processor, the instructions further cause the apparatus to obtain, based on the color depth image, the preset presentation angle.

13. A vehicle comprising:

a video call system comprising:

a processor configured to:

obtain, based on a face image of a user in a video call process, a key feature point of a facial expression of the user, wherein the face image comprises N infrared images of the user and a color face image of the user, and wherein N is an integer greater than 0;

obtain, based on the N infrared images and the color face image, a three-dimensional (3D) head point cloud information of the user;

construct, based on the 3D head point cloud information, a 3D head image of the user;

drive, using the key feature point, the 3D head image to obtain a first target 3D avatar of the user, wherein the first target 3D avatar of the user has an expression of the user;

rotate, based on a preset presentation angle, the first target 3D avatar to obtain a second target 3D avatar at the preset presentation angle; and

transmit the second target 3D avatar; and

a communications apparatus coupled to the processor and configured to:

receive the second target 3D avatar; and

send, to a peer video call device, the second target 3D avatar.

14. The vehicle of claim 13 , wherein the processor is further configured to:

obtain, based on the color face image, a face texture feature of the user; and

further construct, based on the face texture feature, the 3D head image that is a color image, wherein the 3D head point cloud information comprises first 3D head point cloud information of the user or second 3D head point cloud information of the user.

15. The vehicle of claim 13 , wherein the processor is further configured to:

input the color face image and the N infrared images into a feature extraction model to obtain 3D head point cloud information of the user and a face texture feature of the user; and

further construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

16. The vehicle of claim 15 , wherein the feature extraction model comprises a 3D head feature extraction network and a texture feature extraction network, and wherein the processor is further configured to:

input the color face image and the N infrared images into the 3D head feature extraction network to obtain the 3D head point cloud information; and

input the color face image into the texture feature extraction network to obtain the face texture feature.

17. The vehicle of claim 16 , wherein the 3D head feature extraction network is a neural network that uses an encoder-decoder architecture, and wherein the processor is further configured to:

obtain, based on the color face image and the N infrared images, N image pairs, wherein each of the N image pairs comprises the color face image and an infrared image of the user from the N infrared images; and

input the N image pairs into the neural network to obtain the 3D head point cloud information.

18. The vehicle of claim 13 , wherein the processor is further configured to obtain, based on the N infrared images, the preset presentation angle.

19. The vehicle of claim 13 , wherein the video call system further comprises an infrared camera coupled to the processor and configured to:

obtain the N infrared images of the user; and

transmit the N infrared images to the processor.

20. The vehicle of claim 13 , wherein the processor is further configured to:

obtain, based on a color depth image, 3D head point cloud information of the user and a face texture feature of the user; and

construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

21. The vehicle of claim 20 , wherein the processor is further configured to obtain, based on the color depth image, the preset presentation angle.

Assignments (2)

Priority Claims (1)

Continuity (2)

References Cited (32)