Gaze point calculation method, apparatus and device

1. A gaze point calculation method, comprising:

determining a 3D model of a first scene;

obtaining a first panoramic image captured in the first scene, wherein a face image of a face is a part of the first panoramic image;

determining a spatial location of the face according to the first panoramic image;

determining a gaze direction corresponding to the face according to the first panoramic image; and

determining an eye gaze target according to the 3D model, the spatial location and the gaze direction,

wherein the determining the spatial location of the face according to the first panoramic image comprises:

determining angular coordinates of the face relative to a first photographing device, wherein the first photographing device is a device that captures the first panoramic image;

determining a radial distance of the face relative to the first photographing device; and

determining the spatial location according to the angular coordinates and the radial distance;

wherein the determining the eye gaze target according to the 3D model, the spatial location and the gaze direction comprises:

determining an eye gaze area in the 3D model according to the 3D model, the spatial location and the gaze direction; and

determining the eye gaze target according to the eye gaze area;

wherein the determining the eye gaze target according to the eye gaze area comprises:

determining distribution densities corresponding to each sub-area of the eye gaze area, wherein the distribution densities are used to indicate distribution densities of gaze points in the eye gaze area; and

determining the eye gaze target according to the distribution densities and objects included in the 3D model.

2. The method according to claim 1 , wherein the determining the angular coordinates of the face relative to the first photographing device comprises:

determining face detection areas in the first panoramic image;

obtaining first perspective images corresponding to each face detection area;

performing a face detection on the first perspective images, to determine a first coordinate of a face image in each first perspective image, wherein the first coordinate is a coordinate of the face image in a corresponding first perspective image; and

determining angular coordinates of a face corresponding to a face image in each first perspective image relative to the first photographing device according to the first coordinate of the face image in each first perspective image.

3. The method according to claim 1 , wherein the determining the eye gaze area in the 3D model according to the 3D model, the spatial location and the gaze direction comprises:

determining an eye gaze model according to the gaze direction and the spatial location; and

determining an eye gaze area in the 3D model according to the 3D model and the eye gaze model.

4. The method according to claim 1 , wherein the determining the 3D model of the first scene comprises:

determining areas for reconstructing a scene in a second scene image;

obtaining a second perspective image corresponding to each area for reconstructing the scene;

performing a single-image 3D reconstruction on the second perspective images, to obtain depth maps of the second perspective images; and

determining the 3D model according to the depth maps of the second perspective images.

5. A non-transitory readable storage medium, comprising a program or an instruction, when the program or instruction is run on a computer, the gaze point calculation method according to claim 1 is executed.

6. A data processing apparatus, comprising a processor, wherein the processor is coupled with a memory;

the memory is configured to store a computer program; and

the processor, when executing the computer program stored in the memory, is configured to:

determine a 3D model of a first scene;

obtain a first panoramic image captured in the first scene, wherein the first panoramic image comprises a face image of a face;

determine a spatial location of the face according to the first panoramic image;

determine a gaze direction corresponding to the face according to the first panoramic image; and

determine an eye gaze target according to the 3D model, the spatial location and the gaze direction,

wherein when the determine the spatial location of the face according to the first panoramic image is performed, the processor is further configured to:

determine angular coordinates of the face relative to a first photographing device, wherein the first photographing device is a device that captures the first panoramic image;

determine a radial distance of the face relative to the first photographing device; and

determine the spatial location according to the angular coordinates and the radial distance;

wherein the determine the eye gaze target according to the 3D model, the spatial location and the gaze direction is performed, the processor is further configured to:

determine an eye gaze area in the 3D model according to the 3D model, the spatial location and the gaze direction; and

determine the eye gaze target according to the eye gaze area;

wherein when the determine the eye gaze target according to the eye gaze area is performed, the processor is further configured to:

determine distribution densities corresponding to each sub-area of the eye gaze area, wherein the distribution densities are used to indicate distribution densities of gaze points in the eye gaze area; and

determine the eye gaze target according to the distribution densities and objects included in the 3D model.

7. The data processing apparatus according to claim 6 , wherein when the determine the angular coordinates of the face relative to the first photographing device is performed, the processor is further configured to:

determine face detection areas in the first panoramic image;

obtain first perspective images corresponding to each face detection area;

perform a face detection on the first perspective images, to determine a first coordinate of a face image in each first perspective image, wherein the first coordinate is a coordinate of the face image in a corresponding first perspective image; and

determine angular coordinates of a face corresponding to a face image in each first perspective image relative to the first photographing device according to the first coordinate of the face image in the each first perspective image.

8. The data processing apparatus according to claim 6 , wherein when the determine the eye gaze area in the 3D model according to the 3D model, the spatial location and the gaze direction is performed, the processor is further configured to:

determine an eye gaze model according to the gaze direction and the spatial location; and

determine an eye gaze area in the 3D model according to the 3D model and the eye gaze model.

9. The data processing apparatus according to claim 6 , wherein when the determine the 3D model of the first scene is performed, the processor is further configured to:

determine areas for reconstructing a scene in a second scene image;

obtain a second perspective image corresponding to each area for reconstructing the scene;

perform a single-image 3D reconstruction on the second perspective images, to obtain depth maps of the second perspective images; and

determine the 3D model according to the depth maps of the second perspective images.

10. A gaze point calculation method, comprising:

determining a 3D model of a first scene;

obtaining a first panoramic image captured in the first scene, wherein a face image of a face is a part of the first panoramic image;

determining a spatial location of the face according to the first panoramic image;

determining a gaze direction corresponding to the face according to the first panoramic image; and

determining an eye gaze target according to the 3D model, the spatial location and the gaze direction,

wherein the determining the eye gaze target according to the 3D model, the spatial location and the gaze direction comprises:

determining an eye gaze area in the 3D model according to the 3D model, the spatial location and the gaze direction; and

determining the eye gaze target according to the eye gaze area,

wherein the determining the eye gaze target according to the eye gaze area comprises:

determining distribution densities corresponding to each sub-area of the eye gaze area, wherein the distribution densities are used to indicate distribution densities of gaze points in the eye gaze area; and

determining the eye gaze target according to the distribution densities and objects included in the 3D model.