Gaze tracking method, apparatus, device, and medium

1 . A gaze tracking method, applied to a head-mounted device, wherein the head-mounted device comprises a display screen and a camera, and the method comprises:

extracting a plurality of feature points corresponding to a target content displayed on the display screen;

determining first position information of the plurality of feature points on the display screen;

acquiring an image of a user's eye captured by the camera, wherein the image of the user's eye comprises a reflection image of the target content displayed on the display screen on an ocular cornea of the user's eye;

determining a plurality of mapping points corresponding to the plurality of feature points in the image of the user's eye;

determining second position information of the plurality of the mapping points in the image of the user's eye; and

determining a position and an orientation of the user's eye based on the first position information and the second position information.

2 . The method according to claim 1 , wherein determining the plurality of the mapping points corresponding to the plurality of feature points in the image of the user's eye comprises:

determining a first feature of each of the plurality of feature points based on the target content;

extracting an image feature of the image of the user's eye, and determining a second feature of a pixel point contained in the image of the user's eye according to the image feature; and

determining a mapping point matched with the first feature from the pixel point contained in the image of the user's eye according to the second feature.

3 . The method according to claim 2 , wherein the display screen is divided into a plurality of regions, and determining the mapping point matched with the first feature from the pixel point contained in the image of the user's eye according to the second feature comprises:

determining a target region, to which the each feature point belongs, from the plurality of the regions, and determining a sub-region corresponding to the target region in the image of the user's eye; and

matching the first feature with the second feature of each pixel point in the sub-region, and determining the mapping point corresponding to the each feature point from each pixel point according to a matching result.

4 . The method according to claim 2 , wherein the camera is a color camera or a black and white camera, and extracting the image feature of the image of the user's eye comprises:

extracting a color feature, a shape feature, and a corner feature of the image of the user's eye, or

extracting a grayscale feature, a shape feature, and a corner feature of the image of the user's eye.

5 . The method according to claim 1 , wherein determining the position and the orientation of the user's eye based on the first position information and the second position information comprises:

determining third position information of the each feature point relative to the camera according to a position of the camera relative to the display screen and the first position information; and

processing the second position information and the third position information based on a corneal reflection method, and determining the position and the orientation of the user's eye.

6 . The method according to claim 1 , wherein determining the position and the orientation of the user's eye based on the first position information and the second position information comprises:

in a case where a count of the plurality of the feature points is greater than a threshold value, the plurality of the feature points is divided into a plurality of groups of feature point combinations, wherein each of the plurality of groups of feature point combinations comprises a specified number of feature points;

determining a plurality of groups of candidate results according to the first position information of each feature point in the plurality of groups of feature point combinations and the second position information of each mapping point corresponding to the each feature point; and

performing fusion processing according to the plurality of groups of the candidate results, to generate the position and the orientation of the user's eye.

7 . An electronic device, comprising:

a processor; and

a memory for storing an instruction executable by the processor;

wherein the processor is configured to read the executable instruction from the memory and execute the instruction to achieve the gaze tracking method according to claim 1 .

8 . A non-transitory computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the gaze tracking method according to claim 1 is achieved.

9 . A non-transitory computer program product, wherein the non-transitory computer program product comprises a computer program/instruction;

when the computer program/instruction is executed by a processor, the gaze tracking method according to claim 1 is achieved.

10 . A gaze tracking apparatus, applied to a head-mounted device, wherein the head-mounted device comprises a display screen and a camera, and the apparatus comprises:

an extracting module, configured to extract a plurality of feature points corresponding to a target content displayed on the display screen, and determine first position information of the plurality of the feature points on the display screen;

an acquiring module, configured to acquire an image of a user's eye captured by the camera, wherein the image of the user's eye comprises a reflection image of the target content displayed on the display screen on an ocular cornea;

a determining module, configured to determine a plurality of mapping points corresponding to the plurality of feature points in the image of the user's eye, and determine second position information of the plurality of the mapping points in the image of the user's eye; and

a tracking module, configured to determine a position and an orientation of the user's eye according to the first position information and the second position information.