Communication method, wearable device, and storage medium

1 . A communication method, applied to a wearable device, comprising:

obtaining a first characteristic data collected by an eye tracking camera of the wearable device and a second characteristic data collected by an external camera other than the eye tracking camera, wherein the first characteristic data represents data associated with eyes of a wearer of the wearable device, and the second characteristic data represents data associated with a face and a body of the wearer;

based on the first characteristic data and the second characteristic data, performing image reconstruction on the wearer to obtain a reconstructed image, and the reconstructed image being configured to represent a virtual image of the wearer; and

sending the reconstructed image to a communication device in communication with the wearable device, to allow the wearable device and the communication device to be in communication based on the reconstructed image;

wherein the first characteristic data comprises an eye characteristic data, the second characteristic data comprises a facial characteristic data and a body characteristic data, and the performing image reconstruction on the wearer to obtain the reconstructed image based on the first characteristic data and the second characteristic data comprises:

performing feature fusion on the eye characteristic data and the facial characteristic data to obtain a facial fusion data;

based on the facial fusion data and the body characteristic data, obtaining a model-driven data; and

based on the model-driven data and a voice data collected by the wearable device, obtaining the reconstructed image;

wherein the performing feature fusion on the eye characteristic data and the facial characteristic data to obtain the facial fusion data, comprises:

aligning the facial characteristic data with a standard facial data to obtain a facial alignment data;

based on an initial eye data in the facial alignment data and the eye characteristic data, obtaining an eye correction scale;

correcting the eye characteristic data based on the eye correction scale to obtain an eye correction data; and

fusing the eye correction data into the facial alignment data to obtain the facial fusion data.

2 . The method according to claim 1 , wherein the aligning the facial characteristic data with the standard facial data to obtain the facial alignment data comprises:

based on a plurality of facial key points of the facial characteristic data and a plurality of standard key points of the standard facial data, obtaining an overall optimization error of facial alignment; and

performing nonlinear optimization on the overall optimization error to obtain the facial alignment data.

3 . The method according to claim 1 , wherein the obtaining the eye correction scale based on the initial eye data in the facial alignment data and the eye characteristic data comprises:

obtaining a relative external parameter of the eye tracking camera and the external camera;

based on the relative external parameter, a center coordinate of two eye key points contained in the eye characteristic data, and a preset depth distance of the eye tracking camera relative to the eye key point, obtaining an actual distance between the two eye key points; and

based on a ratio of an initial distance between two eye key points in the initial eye data to the actual distance, obtaining the eye correction scale.

4 . The method according to claim 3 , wherein the obtaining the actual distance between the two eye key points based on the relative external parameter, the center coordinate of the two eye key points contained in the eye characteristic data, and the preset depth distance of the eye tracking camera relative to the eye key point, comprises:

based on the center coordinate of the two eye key points and the depth distance, obtaining three-dimensional coordinates of the two eye key points; and

based on the three-dimensional coordinates of the two eye key points and the relative external parameter, obtaining the actual distance between the two eye key points.

5 . The method according to claim 1 , wherein the model-driven data comprises an expression weight and coordinates of a plurality of body key points, and the obtaining the model-driven data based on the facial fusion data and the body characteristic data comprises:

based on a target expression represented by the facial fusion data, a plurality of preset benchmark expressions, and weights of the plurality of benchmark expressions, obtaining an expression weight corresponding to the target expression; and

mapping the plurality of body key points of the body characteristic data to a three-dimensional space to obtain the coordinates of the plurality of body key points.

6 . The method according to claim 1 , wherein the obtaining the reconstructed image based on the model-driven data and the voice data collected by the wearable device comprises:

obtaining a virtual drive model; and

based on the model-driven data and the voice data collected by the wearable device, driving the virtual drive model to obtain the reconstructed image.

7 . The method according to claim 1 , wherein the obtaining the first characteristic data collected by the eye tracking camera of the wearable device and the second characteristic data collected by the external camera other than the eye tracking camera, comprises:

sending a clock synchronization signal to a camera device where the external camera is located or receiving the clock synchronization signal sent by the camera device, the clock synchronization signal being configured for the clock synchronization of the eye tracking camera and the external camera; and

performing collection based on the eye tracking camera and the external camera after the clock synchronization to obtain the first characteristic data and the second characteristic data.

8 . A wearable device, comprising:

a processor; and

a memory for storing instructions executable by the processor;

wherein the processor is configured to:

obtain a first characteristic data collected by an eye tracking camera of the wearable device and a second characteristic data collected by an external camera other than the eye tracking camera, wherein the first characteristic data represents data associated with eyes of a wearer of the wearable device, and the second characteristic data represents data associated with a face and a body of the wearer;

based on the first characteristic data and the second characteristic data, perform image reconstruction on the wearer to obtain a reconstructed image, and the reconstructed image being configured to represent a virtual image of the wearer; and

send the reconstructed image to a communication device in communication with the wearable device, to allow the wearable device and the communication device to be in communication based on the reconstructed image;

wherein the first characteristic data comprises an eye characteristic data, the second characteristic data comprises a facial characteristic data and a body characteristic data, and the processor is further configured to:

perform feature fusion on the eye characteristic data and the facial characteristic data to obtain a facial fusion data;

based on the facial fusion data and the body characteristic data, obtain a model-driven data; and

based on the model-driven data and a voice data collected by the wearable device, obtain the reconstructed image;

wherein the processor is further configured to:

align the facial characteristic data with a standard facial data to obtain a facial alignment data;

based on an initial eye data in the facial alignment data and the eye characteristic data, obtain an eye correction scale;

correct the eye characteristic data based on the eye correction scale to obtain an eye correction data; and

fuse the eye correction data into the facial alignment data to obtain the facial fusion data.

9 . The wearable device according to claim 8 , wherein the processor is further configured to:

based on a plurality of facial key points of the facial characteristic data and a plurality of standard key points of the standard facial data, obtain an overall optimization error of facial alignment; and

perform nonlinear optimization on the overall optimization error to obtain the facial alignment data.

10 . The wearable device according to claim 8 , wherein the processor is further configured to:

obtain a relative external parameter of the eye tracking camera and the external camera;

based on the relative external parameter, a center coordinate of two eye key points contained in the eye characteristic data, and a preset depth distance of the eye tracking camera relative to the eye key point, obtain an actual distance between the two eye key points; and

based on a ratio of an initial distance between two eye key points in the initial eye data to the actual distance, obtain the eye correction scale.

11 . The wearable device according to claim 10 , wherein the processor is further configured to:

based on the center coordinate of the two eye key points and the depth distance, obtain three-dimensional coordinates of the two eye key points; and

based on the three-dimensional coordinates of the two eye key points and the relative external parameter, obtain the actual distance between the two eye key points.

12 . The wearable device according to claim 8 , wherein the model-driven data comprises an expression weight and coordinates of a plurality of body key points, and the processor is further configured to:

based on a target expression represented by the facial fusion data, a plurality of preset benchmark expressions, and weights of the plurality of benchmark expressions, obtain an expression weight corresponding to the target expression; and

map the plurality of body key points of the body characteristic data to a three-dimensional space to obtain the coordinates of the plurality of body key points.

13 . The wearable device according to claim 8 , wherein the processor is further configured to:

obtain a virtual drive model; and

based on the model-driven data and the voice data collected by the wearable device, drive the virtual drive model to obtain the reconstructed image.

14 . The wearable device according to claim 8 , wherein the processor is further configured to:

send a clock synchronization signal to a camera device where the external camera is located or receiving the clock synchronization signal sent by the camera device, the clock synchronization signal being configured for the clock synchronization of the eye tracking camera and the external camera; and

perform collection based on the eye tracking camera and the external camera after the clock synchronization to obtain the first characteristic data and the second characteristic data.

15 . A non-transitory computer-readable storage medium, storing instructions thereon, wherein in case that the instructions are executed by a processor of a wearable device, the wearable device is enabled to:

obtain a first characteristic data collected by an eye tracking camera of the wearable device and a second characteristic data collected by an external camera other than the eye tracking camera, wherein the first characteristic data represents data associated with eyes of a wearer of the wearable device, and the second characteristic data represents data associated with a face and a body of the wearer;

based on the first characteristic data and the second characteristic data, perform image reconstruction on the wearer to obtain a reconstructed image, and the reconstructed image being configured to represent a virtual image of the wearer; and

send the reconstructed image to a communication device in communication with the wearable device, to allow the wearable device and the communication device to be in communication based on the reconstructed image;

wherein the first characteristic data comprises an eye characteristic data, the second characteristic data comprises a facial characteristic data and a body characteristic data, and the wearable device is configured to:

perform feature fusion on the eye characteristic data and the facial characteristic data to obtain a facial fusion data;

based on the facial fusion data and the body characteristic data, obtain a model-driven data; and

based on the model-driven data and a voice data collected by the wearable device, obtain the reconstructed image;

wherein the wearable device is further configured to:

align the facial characteristic data with a standard facial data to obtain a facial alignment data;

based on an initial eye data in the facial alignment data and the eye characteristic data, obtain an eye correction scale;

correct the eye characteristic data based on the eye correction scale to obtain an eye correction data; and

fuse the eye correction data into the facial alignment data to obtain the facial fusion data.