Electronic device identifying direction of gaze and method for operating the same
An electronic device may include a camera and at least one processor. The at least one processor may be configured to obtain an eye image of a user through the camera, obtain pupil data and a retina image from the obtained eye image, obtain an eye model including the retina image, based on the obtained pupil data and the obtained retina image, and identify at least one of a position of an eye or a gaze direction, based on the obtained eye model.
1 . An electronic device comprising:
a camera; and
at least one processor configured to:
obtain an eye image of an eye of a user from the camera,
obtain pupil data and a retina image from the eye image,
obtain, based on the pupil data and the retina image, an eye model comprising the retina image, and
identify, based on the eye model, at least one of a position of the eye or a gaze direction.
2 . The electronic device of claim 1 , further comprising a light source configured to emit light of a preset wavelength band to the eye.
3 . The electronic device of claim 1 , wherein the at least one processor is further configured to:
receive a plurality of eye images, comprising the eye image, according to rotation of the eye from the camera,
obtain a plurality of pupil data, comprising the pupil data, and retina images, comprising the retina image, from the plurality of eye images, and
obtain, based on the plurality of pupil data and the retina images, the eye model.
4 . The electronic device of claim 3 , wherein obtaining the retina images comprises obtaining a plurality of images of a retina inside an eyeball of the eye of the user, the plurality of images of the retina comprising a first image, of a first area of the retina, and a second image of a second area of the retina, the first area of the retina being at least partly different than the second area of the retina.
5 . The electronic device of claim 4 , wherein
the plurality of pupil data being of a pupil of the eye of the user and comprising first pupil data and second pupil data,
the first pupil data indicating a first position of the pupil, a first shape of the pupil, a first rotation direction of the eye, and a first rotation angle of the eye,
the second pupil data indicating a second position of the pupil, a second shape of the pupil, a second rotation direction of the eye, and a second rotation angle of the eye,
at least one of:
the first position of the pupil is different than the second position of the pupil,
the first shape of the pupil is different than the second shape of the pupil,
the first rotation direction of the eye is different than the second rotation direction of the eye, and
the first rotation angle of the eye is different than the second rotation angle of the eye,
the eye model is a three-dimensional (3D) eye model in which at least the first area of the retina and the second area of the retina are modeled and mapped to ones of the first pupil data and the second pupil data with:
the first area of the retina mapped to the first pupil data, and
the second area of the retina mapped to the second pupil data.
6 . The electronic device of claim 3 , further comprising a display,
wherein the at least one processor is further configured to sequentially display an object inducing the gaze of the user to different positions on the display.
7 . The electronic device of claim 1 , wherein the at least one processor is further configured to identify area of pupil from the eye image and obtain at least one of a position of a pupil, a shape of the pupil, or the gaze direction based on the area of the pupil.
8 . The electronic device of claim 7 , wherein the at least one processor is further configured to obtain, based on the at least one of the position of the pupil, the shape of the pupil, or the gaze direction, any one of the position of the eye or a shape of the eye.
9 . The electronic device of claim 1 , wherein the at least one processor is further configured to obtain the eye model by mapping the retina image to the pupil data.
10 . The electronic device of claim 1 , wherein the at least one processor is further configured to identify, based on the eye model, the at least one of the position of the eye or the gaze direction corresponding to the eye image.
11 . The electronic device of claim 10 , wherein the at least one processor is further configured to, based on the retina image, determine whether the eye image corresponds to a pre-stored eye image of the user corresponding to a pre-stored eye model.
12 . The electronic device of claim 10 , wherein the at least one processor is further configured to:
identify, from the eye image, area of a pupil corresponding to the pupil; and
obtain, based on the area of the pupil, the gaze direction.
13 . The electronic device of claim 12 , wherein the at least one processor is further configured to, based on the retina image and the eye model, detect whether the position of the eye (E) is changed.
14 . The electronic device of claim 13 , wherein the at least one processor is further configured to, based on the retina image and the eye model, obtain the position of the eye, when detecting the change in the position of the eye.
15 . A method for operating an electronic device, the method comprising:
obtaining an eye image of an eye of a user from a camera,
obtaining pupil data and a retina image from the eye image,
obtaining, based on the pupil data and the retina image, an eye model comprising the retina image, and
identifying, based on the eye model, at least one of a position of the eye or a gaze direction.
16 . The method of claim 15 , wherein the obtaining the eye image comprises receiving, from the camera, a plurality of eye images, comprising the eye image, according to a rotation of the eye,
wherein the obtaining the pupil data and the retina image comprises obtaining a plurality of pupil data, comprising the pupil data, and retina images, comprising the retina image, from the plurality of eye images, and
wherein the obtaining the eye model comprises obtaining, based on the plurality of pupil data and the retina images, the eye model.
17 . The method of claim 15 , wherein the obtaining the pupil data and the retina image comprises identifying area of a pupil from the eye image and obtaining, based on the area of the pupil, at least one of a position of the pupil, a shape of the pupil, or the gaze direction, and
wherein the obtaining the eye model comprises obtaining, based on the at least one of the position of the pupil, the shape of the pupil, or the gaze direction any one of the position of the eye or a shape of the eye.
18 . The electronic device of claim 1 , wherein the retina image is an image of a retina inside of an eyeball of the eye of the user.
19 . The electronic device of claim 18 , wherein
the pupil data indicates at least one of a position of a pupil of the eye of the user, a shape of the pupil, a rotation direction of the eye, and a rotation angle of the eye, and
the eye model is a three-dimensional (3D) eye model in which the image of the retina is mapped to at least one of the position of the pupil, the shape of the pupil, the rotation direction of the eye, and the rotation angle of the eye.