Visual line detection device and visual line detection program
[Technical Problem] To provide a gaze detection apparatus even when multiple bright spots cannot be observed on the cornea. Solution to Problem In this gaze detection device, when the bright spots observed on the cornea of the eye based on light emitted from the plurality of light sources are in a first state, the processor: (1-1) identifies the position of the center of the cornea based on multiple bright spots, (1-2) identifies the position of the pupil of the eye based on the image of the imaging device, and (1-3) identifies the gaze direction based on the center of the cornea and the pupil position. On the other hand, when the bright spots observed on the cornea of the eye based on the plurality of light sources are in the second state, the processor: (2-1) identifies the shape of the eye according to the data on the radius of the eye and the center of the eye stored in the memory unit, and (2-2) identifies the gaze direction from the image of the imaging device and the center position of the eyeball.
1 . A gaze detection apparatus comprising:
a plurality of light sources illuminating an eye;
an imaging device for imaging an image of the eye; and
a processor,
wherein the processor:
if a state of a bright spot observed on a cornea of the eye based on the plurality of light sources is in a first state,
(1-1) identifies a position of a center of the cornea based on a plurality of the bright spots;
(1-2) identifies a position of a pupil of the eye based on the image of the imaging device; and
(1-3) identifies a gaze direction based on the position of the center of the cornea and the position of the pupil, and
if the state of the bright spot observed on the cornea of the eye based on the plurality of light sources is in a second state,
(2-1) identifies a shape of an eyeball according to data of a radius of the eyeball and a position of a center of the eyeball stored in a storage unit; and
(2-2) identifies the gaze direction from the image of the imaging device and the position of the center of the eyeball.
2 . The gaze detection apparatus according to claim 1 , wherein the processor is configured to identify and store a radius of the eyeball and a position of the center of the eyeball according to data of the gaze directions acquired at a plurality of timings.
3 . The gaze detection apparatus according to claim 1 , wherein the processor identifies the position of the center of the eyeball by finding an intersection of a plurality of specified gaze, and defines the radius of the eyeball based on a difference between a center position of the eye and a position of the pupil.
4 . The gaze detection apparatus according to claim 1 , wherein the processor, in the second state, identifies the position of the pupil from the image of the imaging device and defines the gaze direction based on the position of the center of the eyeball and the position of the pupil.
5 . The gaze detection apparatus according to claim 1 , wherein, when the processor, in the first state, fails to identify the position of the pupil of the eye, the processor identifies the gaze direction according to the shape of the eye and the position of the center of the cornea.
6 . The gaze detection apparatus according to claim 1 , wherein the processor, in the second state, identifies the direction of a beam emitted from the eye from an image of the imaging device, and identifies the gaze direction based on a center position of the eye and the direction of the beam.
7 . A non-transitory computer-readable medium storing a gaze detection program for detecting a gaze direction of an eye, the program comprising instructions that, when executed by a processor of a computer system, cause the computer system to perform operations comprising:
illuminating the eye from a plurality of light sources, and determining a state of a bright spot observed on a cornea of an eye;
if the state is in a first state,
(1-1) identifying a position of a center of the cornea based on a plurality of the bright spots;
(1-2) identifying a position of a pupil of the eye based on the image of the eye,
(1-3) identifying a gaze direction based on the position of the center of the cornea and the position of the pupil;
on the other hand, if the state of the bright spot observed on the cornea of the eye based on the plurality of light sources is a second state,
(2-1) identifying a shape of an eyeball according to data of a radius of the eyeball stored in a storage unit and the position of the center of the eyeball; and
(2-2) identifying the gaze direction from the image of the eye and the position of the center of the eyeball.
8 . The non-transitory computer-readable medium of claim 7 , wherein a radius of the eyeball and the position of the center of the eyeball is identified and stored in the storage unit.
9 . The non-transitory computer-readable medium of claim 7 , wherein the position of the center of the eyeball is identified by computing an intersection of a identified gazes, and the radius of the eyeball is identified based on a difference between the position of the center of the eyeball and a position of the pupil.
10 . The non-transitory computer-readable medium of claim 7 , wherein in the second state, the position of the pupil is located from the image of the imaging device and the gaze direction is identified based on the position of the center of the eyeball and the position of the pupil.
11 . The non-transitory computer-readable medium program of claim 7 , wherein in the first state, when the position of the pupil of the eye fails to be identified, the gaze direction is identified in accordance with the shape of the eye and the position of the center of the cornea.
12 . The non-transitory computer-readable medium of claim 7 , wherein in the second state, the direction of the beam emitted from the eye from the image of the imaging device is identified and the gaze direction is identified based on the position of the center of the eyeball and the direction of the beam.
13 . A head mount display comprising: a housing mounted on a head of a user; and a gaze detection device mounted on the housing for detecting a gaze direction of the user,
wherein the gaze detection device comprises:
a plurality of light sources that illuminate an eye of the user;
an imaging device for imaging an image of the eye; and
a processor,
wherein the processor:
if bright spots observed on a cornea of the eye based on the plurality of light sources are in a first state,
(1-1) identifies a position of a center of the cornea based on the bright spots;
(1-2) identifies a position of a pupil of the eye based on an image of the imaging device; and
(1-3) identifies the gaze direction based on the position of the center of the cornea and the position of the pupil, and
if the bright spots observed on the cornea of the eye based on the plurality of light sources are in a second state,
(2-1) identifies a shape of the eye according to data of a radius of the eye and the position of the center of the eye stored in the storage section; and
(2-2) identifies the gaze direction from the image of the imaging device and the position of the center of the eyeball.
14 . The gaze detection apparatus according to claim 1 , wherein:
the first state corresponds to a state in which a number of bright spots observed on the cornea of the eye based on the plurality of light sources is equal to or greater than a number required to identify the position of the center of the cornea, and
the second state corresponds to a state in which the number of the bright spots observed on the cornea of the eye based on the plurality of light sources is less than the number required to identify the position of the center of the cornea.
15 . The gaze detection apparatus according to claim 14 , wherein:
when the number of the bright spots observed on the cornea of the eye based on the plurality of light sources is equal to or greater than the number required to identify the position of the center of the cornea, the processor further identifies the position of the center of the eyeball and the radius of the eyeball based on the identified gaze direction, and stores the identified position of the center of the eyeball and the radius of the eyeball in the storage unit.
16 . The computer-readable medium according to claim 7 , wherein:
the first state corresponds to a state in which a number of bright spots observed on the cornea of the eye based on the plurality of light sources is equal to or greater than a number required to identify the position of the center of the cornea, and
the second state corresponds to a state in which the number of the bright spots observed on the cornea of the eye based on the plurality of light sources is less than the number required to identify the position of the center of the cornea.
17 . The computer-readable medium according to claim 16 , wherein:
when the number of the bright spots observed on the cornea of the eye based on the plurality of light sources is equal to or greater than the number required to identify the position of the center of the cornea, the processor further identifies the position of the center of the eyeball and the radius of the eyeball based on the identified gaze direction, and stores the identified position of the center of the eyeball and the radius of the eyeball in the storage unit.
18 . The head mount display according to claim 13 , wherein:
the first state corresponds to a state in which a number of bright spots observed on the cornea of the eye based on the plurality of light sources is equal to or greater than a number required to identify the position of the center of the cornea, and
the second state corresponds to a state in which the number of the bright spots observed on the cornea of the eye based on the plurality of light sources is less than the number required to identify the position of the center of the cornea.
19 . The head mount display according to claim 18 , wherein:
when the number of the bright spots observed on the cornea of the eye based on the plurality of light sources is equal to or greater than the number required to identify the position of the center of the cornea, the processor further identifies the position of the center of the eyeball and the radius of the eyeball based on the identified gaze direction, and stores the identified position of the center of the eyeball and the radius of the eyeball in the storage unit.