Device and method for tracking eyeballs, and display device

1 . A device for tracking eyeballs, comprising: a support, a control assembly, a light filling assembly, an image acquisition assembly, and at least two eyeball tracking cameras; wherein

the at least two eyeball tracking cameras are disposed on at least two different positions of the support, and at least parts of shooting ranges of the at least two eyeball tracking cameras are not overlapped;

the light filling assembly and the image acquisition assembly are disposed on the support, wherein the image acquisition assembly is disposed on a lower portion of the support, the shooting ranges of the at least two eyeball tracking cameras are within a shooting range of the image acquisition assembly, and a gap is defined between edges of the shooting ranges of the at least two eyeball tracking cameras and an edge of the shooting range of the image acquisition assembly, wherein a size of the gap is greater than a size of a predetermined human face, the image acquisition assembly comprises a color camera, and the eyeball tracking camera comprises an infrared camera;

the control assembly is electrically connected to the at least two eyeball tracking cameras and the image acquisition assembly, and is configured to determine a gaze position of the eyeballs within the shooting ranges of the at least two eyeball tracking cameras;

the control assembly is configured to determine the gaze position, within the shooting ranges of the at least two eyeball tracking cameras, of the eyeballs, whose distance from a to-be-observed face meets a predetermined distance range, on the to-be-observed face;

the at least two eyeball tracking cameras are disposed at a lower portion of the support and arranged in a horizontal direction, optical axes of the at least two eyeball tracking cameras are parallel and coplanar, a first plane determined based on the optical axes of the at least two eyeball tracking cameras is intersected with a vertical line running through a center of the to-be-observed face, and an intersection point is at a center of the predetermined distance range; and

a field of view of the image acquisition assembly is greater than a field of view of any of the at least two eyeball tracking cameras, and at a position having a smallest vertical distance to the to-be-observed face in the predetermined distance range and at a position having a largest vertical distance to the to-be-observed face in the predetermined distance range, regions capable of being shot by the image acquisition assembly are greater than regions capable of being shot by the at least two eyeball tracking cameras.

2 . The device for tracking eyeballs according to claim 1 , wherein in the at least two eyeball tracking cameras, an overlapping region is defined between the shooting ranges of two adjacent eyeball tracking cameras.

3 . The device for tracking eyeballs according to claim 2 , wherein a size of the overlapping region is greater than a size of predetermined eyeballs.

4 . The device for tracking eyeballs according to claim 1 , wherein the support comprises a mounting portion for disposing the to-be-observed face.

5 . The device for tracking eyeballs according to claim 4 , wherein

the to-be-observed face is a display face of a display panel, and the support comprises the mounting portion for disposing the display panel.

6 . The device for tracking eyeballs according to claim 1 , wherein the light filling assembly comprises at least two light filling lamps disposed on different positions of the support.

7 . The device for tracking eyeballs according to claim 6 , wherein the at least two light filling lamps are arranged in a horizontal direction on the support.

8 . The device for tracking eyeballs according to claim 1 , wherein a resolution of the color camera is less than a resolution of the infrared camera.

9 . A method for tracking eyeballs, applicable to a control assembly of a device for tracking eyeballs, wherein the device for tracking eyeballs comprises: a support, a light filling assembly, an image acquisition assembly, and at least two eyeball tracking cameras; wherein the at least two eyeball tracking cameras are disposed on at least two different positions of the support, and at least parts of shooting ranges of the at least two eyeball tracking cameras are not overlapped; the light filling assembly and the image acquisition assembly are disposed on the support, wherein the image acquisition assembly is disposed on a lower portion of the support, the shooting ranges of the at least two eyeball tracking cameras are within a shooting range of the image acquisition assembly, and a gap is defined between edges of the shooting ranges of the at least two eyeball tracking cameras and an edge of the shooting range of the image acquisition assembly, wherein a size of the gap is greater than a size of a predetermined human face, the image acquisition assembly comprises a color camera, and the eyeball tracking camera comprises an infrared camera; and the control assembly is electrically connected to the at least two eyeball tracking cameras and the image acquisition assembly; wherein the control assembly is configured to determine the gaze position, within the shooting ranges of the at least two eyeball tracking cameras, of the eyeballs, whose distance from a to-be-observed face meets a predetermined distance range, on the to-be-observed face; the at least two eyeball tracking cameras are disposed at a lower portion of the support and arranged in a horizontal direction, optical axes of the at least two eyeball tracking cameras are parallel and coplanar, a first plane determined based on the optical axes of the at least two eyeball tracking cameras is intersected with a vertical line running through a center of the to-be-observed face, and an intersection point is at a center of the predetermined distance range; and a field of view of the image acquisition assembly is greater than a field of view of any of the at least two eyeball tracking cameras, and at a position having a smallest vertical distance to the to-be-observed face in the predetermined distance range and at a position having a largest vertical distance to the to-be-observed face in the predetermined distance range, regions capable of being shot by the image acquisition assembly are greater than regions capable of being shot by the at least two eyeball tracking cameras;

the method comprising:

turning on the light filling assembly;

acquiring a first image acquired by the at least two eyeball tracking cameras; and

determining a gaze position of the eyeballs based on the first image;

wherein prior to determining the gaze position of the eyeballs based on the first image, the method further comprises:

acquiring a second image acquired by the image acquisition assembly;

acquiring facial information based on the second image; and

determining positions of the eyeballs in the second image based on the facial information; and

determining the gaze position of the eyeballs based on the first image comprises:

determining positions of the eyeballs in the first image based on the positions of the eyeballs in the second image; and

determining the gaze position of the eyeballs based on the positions of the eyeballs in the first image and the first image.

10 . The method according to claim 9 , wherein determining the gaze position of the eyeballs based on the positions of the eyeballs in the first image and the first image comprises:

acquiring, based on the positions of the eyeballs in the first image and the first image, positions of pupils of the eyeballs and positions of spots, on the eyeballs, of light emitted by the light filling assembly;

acquiring line-of-sight models of the at least two eyeball tracking cameras, wherein the line-of-sight models of the at least two eyeball tracking cameras are configured to determine the gaze position of the eyeballs based on the positions of the pupils of the eyeballs and the positions of the spots in the first image acquired by the at least two eyeball tracking cameras; and

determining the gaze position of the eyeballs based on the line-of-sight models of the at least two eyeball tracking cameras, the positions of the pupils of the eyeballs, and the positions of the spots.

11 . The method according to claim 10 , wherein determining the gaze position of the eyeballs based on the line-of-sight models of the at least two eyeball tracking cameras, the positions of the pupils of the eyeballs, and the positions of the spots comprises:

determining at least one gaze position of the eyeballs based on the line-of-sight models of the at least two eyeball tracking cameras; and

determining a target gaze position based on the at least one gaze position.

12 . The method according to claim 10 , wherein

the first image is an image acquired by a first eyeball tracking camera in the at least two eyeball tracking cameras, the light filling assembly comprises at least two light filling lamps disposed on different positions of the support, and the first image comprises two eyeballs; and

acquiring, based on the positions of the eyeballs in the first image and the first image, the positions of the pupils of the eyeballs and the positions of the spots, on the eyeballs, of the light emitted by the light filling assembly comprises:

acquiring, based on positions of the two eyeballs in the first image and the first image, positions of pupils of the two eyeballs and positions of spots, on the two eyeballs, of light emitted by the at least two light filling lamps; and

acquiring the line-of-sight models of the at least two eyeball tracking cameras comprises:

determining, based on each sample data, line-of-sight models corresponding to at least one sample data, wherein the sample data comprises a position of a pupil of one of the two eyeballs and a position of a spot on the one of the two eyeballs, and the line-of-sight models corresponding to the at least one sample data are line-of-sight models of the first eyeball tracking camera in the at least two eyeball tracking cameras.

13 . A display device, comprising: a display panel, a housing, and a device for tracking eyeballs, wherein the device for tracking eyeballs comprises: a support, a control assembly, a light filling assembly, an image acquisition assembly, and at least two eyeball tracking cameras;

wherein the light filling assembly, the image acquisition assembly, and the at least two eyeball tracking cameras are disposed on the housing and face towards a light emitting direction of the display panel, the at least two eyeball tracking cameras are disposed on at least two different positions of the housing, and the control assembly is electrically connected to the at least two eyeball tracking cameras and the image acquisition assembly; and

the light filling assembly and the image acquisition assembly are disposed on the support, wherein the image acquisition assembly is disposed on a lower portion of the support, the shooting ranges of the at least two eyeball tracking cameras are within a shooting range of the image acquisition assembly, and a gap is defined between edges of the shooting ranges of the at least two eyeball tracking cameras and an edge of the shooting range of the image acquisition assembly, wherein a size of the gap is greater than a size of a predetermined human face, the image acquisition assembly comprises a color camera, and the eyeball tracking camera comprises an infrared camera;

wherein the control assembly is configured to determine the gaze position, within the shooting ranges of the at least two eyeball tracking cameras, of the eyeballs, whose distance from a to-be-observed face meets a predetermined distance range, on the to-be-observed face;

the at least two eyeball tracking cameras are disposed at a lower portion of the support and arranged in a horizontal direction, optical axes of the at least two eyeball tracking cameras are parallel and coplanar, a first plane determined based on the optical axes of the at least two eyeball tracking cameras is intersected with a vertical line running through a center of the to-be-observed face, and an intersection point is at a center of the predetermined distance range; and

a field of view of the image acquisition assembly is greater than a field of view of any of the at least two eyeball tracking cameras, and at a position having a smallest vertical distance to the to-be-observed face in the predetermined distance range and at a position having a largest vertical distance to the to-be-observed face in the predetermined distance range, regions capable of being shot by the image acquisition assembly are greater than regions capable of being shot by the at least two eyeball tracking cameras.

14 . The display device according to claim 13 , wherein the at least two eyeball tracking cameras are disposed on the housing and under the display panel, and arranged in a horizontal direction.

15 . The display device according to claim 13 , wherein the image acquisition assembly is disposed on the housing and under the display panel, and distances between the image acquisition assembly and two ends of lower edges of the display face are equal.

16 . The display device according to claim 13 , wherein the housing comprises a base and a frame body disposed on the base, and the light filling assembly, the image acquisition assembly, and the at least two eyeball tracking cameras are disposed on the frame body.