Goggle type display device, eye gaze detection method, and eye gaze detection system

1. A goggle type display device comprising a goggle-shaped frame and a display mounted on said frame including:

an image-capturing unit configured to be disposed inside said frame so as to capture a facial image of a user wearing said goggle-shaped frame, the facial image including simultaneously an eye and a ridge of a nose; and

an eye gaze position estimator configured to estimate an eye gaze position indicating a position on said display with which said user has a visual contact, on the basis of tilt of an ellipse of a pupil and an iris with reference to the ridge of the nose included in said facial image.

2. The goggle type display device according to claim 1 , wherein said eye gaze position estimator comprises:

a target site detector configured to detect an image of a pupil or an iris of said eye and an image of said ridge of the nose from said facial image; and

an eye gaze coordinate generator configured to generate coordinates of said eye gaze position which is with reference to said display on the basis of said image of the pupil or the iris and said image of the ridge of the nose.

3. The goggle type display device according to claim 2 , wherein said eye gaze coordinate generator generates coordinates of said eye gaze position on the basis of a shape of said image of the pupil or the iris and a position of said pupil or the iris with reference to said ridge of the nose.

4. The eye gaze detection system according to claim 3 , wherein said shape of said image of the pupil or the iris refers to a width of ellipses of said image of the pupil or the iris.

5. The goggle type display device according to claim 2 , wherein

said target site detector detects images of an upper lid and a lower lid of said eye from said facial image, and

said eye gaze coordinate generator generates coordinates of said eye gaze position on the basis of angles of said upper lid and said lower lid which is with reference to said ridge of the nose.

6. The goggle type display device according to claim 2 , wherein

said goggle-shaped frame includes a pair of temples, and

said image-capturing unit is disposed on an inner side of at least one of said pair of temples.

7. The goggle type display device according to claim 2 , further comprising a second image-capturing unit configured to obtain a second facial image which includes said eye and an outer corner of the eye or a contour of a cheek, wherein

said target site detector detects an image of a pupil or an iris of said eye and an image of said outer corner of the eye or said contour of the cheek from said second facial image, and

said eye gaze coordinate generator generates coordinates of said eye gaze position on the basis of a shape of said image of the pupil or the iris and a position of said pupil or said iris with reference to said outer corner of the eye or said contour of the cheek.

8. The goggle type display device according to claim 7 , wherein

said target site detector detects positions of an upper lid and a lower lid of said eye and a position of said outer corner of the eye or said contour of the cheek from said second facial image, and

said eye gaze coordinate generator generates coordinates of said eye gaze position on the basis of angles of said upper lid and said lower lid which is with reference to said outer corner of the eye or said contour of the cheek.

9. The goggle type display device according to claim 7 , wherein the second image-capturing unit is disposed in a bridge of said goggle-shaped frame.

10. The goggle type display device according to claim 7 , wherein said eye gaze position estimator estimates said eye gaze position by synthesizing the coordinates of the eye gaze position generated on the basis of said facial image and the coordinates of the eye gaze position generated on the basis of said second facial image.

11. An eye gaze detection method in a goggle type display device, the goggle type display device comprising a goggle-shaped frame and a display and capable of being worn on a head of a user, the method including:

a step of photographing a face of the user wearing said frame to obtain a facial image of said user wearing said goggle-shaped frame, the facial image simultaneously including an eye and a ridge of a nose; and

a step of estimating an eye gaze position indicating a position on said display with which said user has a visual contact, on the basis of tilt of an ellipse of a pupil and an iris with reference to the ridge of the nose included in said facial image.

12. The eye gaze detection method according to claim 11 , wherein said step of estimating the eye gaze position comprises:

a step of detecting an image of a pupil or an iris of said eye and an image of said ridge of the nose from said facial image; and

a step of generating coordinates of said eye gaze position which is with reference to said display on the basis of said image of the pupil or the iris and said image of the ridge of the nose.

13. The eye gaze detection method according to claim 12 , wherein in said step of generating coordinates of the eye gaze position, coordinates of the eye gaze position are generated on the basis of a shape of said image of the pupil or the iris and a position of said pupil or said iris which is with reference to said ridge of the nose.

14. The eye gaze detection method according to claim 12 , wherein said step of generating coordinates of the eye gaze position comprises:

a step of detecting images of an upper lid and a lower lid of said eye; and

a step of generating coordinates of the eye gaze position on the basis of angles of said upper lid and said lower lid which are with reference to said ridge of the nose.

15. The eye gaze detection method according to claim 12 , further comprising:

a step of obtaining a second facial image including said eye and an outer corner of the eye or a contour of a cheek; and

a step of detecting an image of said pupil or said iris of the eye and an image of said outer corner of the eye or said contour of the cheek from said second facial image, wherein

in said step of generating coordinates of the eye gaze position, coordinates of the eye gaze position are generated on the basis of a shape of said image of the pupil or the iris and a position of said pupil or said iris which is with reference to said outer corner of the eye or said contour of the cheek.

16. The eye gaze detection method according to claim 15 , wherein said step of generating coordinates of the eye gaze position comprises:

a step of detecting images of an upper lid and a lower lid of said eye and an image of said outer corner of the eye or said contour of the cheek from said second facial image; and

a step of generating said coordinates of the eye gaze position on the basis of angles of said upper lid and said lower lid which are with reference to said outer corner of the eye or said contour of the cheek.

17. The eye gaze detection method according to claim 15 , wherein said step of estimating the eye gaze position comprises a step of synthesizing the coordinates of the eye gaze position generated on the basis of said facial image and the coordinates of the eye gaze position generated on the basis of said second facial image to estimate said eye gaze position.

18. An eye gaze detection system comprising:

an image-capturing unit configured to photograph a facial image of a user wearing a goggle-shaped frame, the facial image simultaneously including an eye and a ridge of a nose;

a target site detector configured to detect an image of said eye and an image of said ridge of the nose from said facial image; and

an eye gaze position detector configured to detect an eye gaze position indicating a position on a display with which said user has a visual contact, on the basis of tilt of an ellipse of a pupil and an iris with reference to the ridge of the nose included in said facial image.

19. The eye gaze detection system according to claim 18 , wherein

said target site detector detects an image of a pupil or an iris of said eye, and

said eye gaze position detector detects said eye gaze position on the basis of a shape of said image of the pupil or the iris and a position of said pupil or said iris with reference to said ridge of the nose.

20. The eye gaze detection system according to claim 18 , wherein

said target site detector detects images of an upper lid and a lower lid of said eye, and

said eye gaze position detector detects said eye gaze position on the basis of angles of said upper lid and said lower lid which are with reference to said ridge of the nose.

21. The eye gaze detection system according to claim 18 , wherein

said image-capturing unit obtains a second facial image including said eye, an outer corner of the eye or a contour of a cheek,

said target site detector detects an image of the pupil or the iris of said eye and an image of said outer corner of the eye or said contour of the cheek from said second facial image, and

said eye gaze position detector detects said eye gaze position on the basis of a shape of said image of the pupil or the iris and a position of said pupil or said iris which is with reference to said outer corner of the eye or said contour of the cheek.

22. The eye gaze detection system according to claim 21 , wherein

said target site detector detects images of an upper lid and a lower lid of said eye from said second facial image, and

said eye gaze position detector detects said eye gaze position on the basis of angles of said upper lid and said lower lid which is with reference to said outer corner of the eye or said contour of the cheek.