IP Library Granted Patent US 10,698,482
Granted Patent B2
US 10,698,482 · App. 16/240,974 · Granted Jun 30, 2020

Gaze tracking using non-circular lights

Inventors: Ville Miettinen (Helsinki, FI); Klaus Melakari (Oulu, FI)
Assignee: Varjo Technologies Oy
G06F3/013G02B27/0093G02B27/017G06K9/00604G02B2027/0138G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,698,482
App. No.
16/240,974
Granted
Jun 30, 2020
Kind
B2
Abstract

Disclosed is gaze-tracking system for head-mounted display apparatus, the gaze-tracking system including a plurality of non-circular light sources that emit light for illuminating a user's eye when the display apparatus is worn by user, at least one camera, and a processor. The processor is configured to control the at least one camera to capture an image of user's eye and reflections of the plurality of non-circular light sources from user's eye; identify at least one of the plurality of non-circular light sources from where at least one of the reflections originated, based upon shapes, rotational orientations and relative positions of the reflections of the plurality of non-circular light sources, to differentiate said reflections from visual artifacts; and detect a gaze direction of user based upon a relative position of a pupil of user's eye with respect to the reflections of the plurality of non-circular light sources.

Claims (33)

1. A gaze-tracking system for use in a head-mounted display apparatus, the gaze-tracking system comprising:

a plurality of non-circular light sources that, in operation, emit light for illuminating a user's eye when the head-mounted display apparatus in operation is worn by the user;

at least one camera; and

a processor configured to:

control the at least one camera to capture an image of the user's eye and reflections of the plurality of non-circular light sources from the user's eye;

identify at least one of the plurality of non-circular light sources from where at least one of the reflections originated, based upon shapes, rotational orientations and relative positions of the reflections of the plurality of non-circular light sources, to differentiate said reflections from visual artifacts; and

detect a gaze direction of the user based upon a relative position of a pupil of the user's eye with respect to the reflections of the plurality of non-circular light sources;

wherein at least two of the plurality of non-circular light sources have a same non-circular shape and the at least two of the plurality of non-circular light sources have different rotational orientations.

2. The gaze-tracking system of claim 1 , wherein the plurality of non-circular light sources are arranged in a substantially circular manner, wherein the reflections of the plurality of non-circular light sources appear in the captured image as a plurality of non-circular glints arranged in a substantially circular manner.

3. The gaze-tracking system of claim 2 , wherein the processor is configured to identify a polygon defined by the plurality of non-circular glints, and to determine the relative position of the user's pupil with respect to the polygon.

4. The gaze-tracking system of claim 2 , wherein the processor is configured to use the plurality of non-circular glints as references for a next image, to differentiate reflections of the plurality of non-circular light sources from visual artifacts in the next image.

5. The gaze-tracking system of claim 1 , wherein a given non-circular light source has a shape that is symmetrical about only one axis and such shape only repeats when the light source is rotated through 360 degrees.

6. The gaze-tracking system of claim 5 , wherein the shape of the given non-circular light source is selected from the group consisting of A, V and Y.

7. The gaze-tracking system of claim 1 , wherein at least two of the plurality of non-circular light sources have different non-circular shapes.

8. The gaze-tracking system of claim 1 , wherein the processor is configured to divide the plurality of non-circular light sources into a plurality of groups, and to control the plurality of groups to emit light in a predefined manner, based upon a time-division multiplexing rule.

9. The gaze-tracking system of claim 1 , wherein the emitted light has an infrared wavelength or a near-infrared wavelength.

10. The gaze-tracking system of claim 1 , wherein the plurality of non-circular light sources are arranged around a periphery of an exit optical element of the head-mounted display apparatus.

11. The gaze-tracking system of claim 10 , further comprising at least one optical element that is substantially transparent for visible light, but is substantially reflective for infrared or near-infrared light, wherein the at least one optical element is positioned on an optical path between at least one image renderer of the head-mounted display apparatus and the exit optical element, and wherein the reflections of the plurality of non-circular light sources, when incident upon the at least one optical element, are reflected towards the at least one camera.

12. A method of tracking a user's gaze, via a gaze-tracking system of a head-mounted display apparatus, the gaze-tracking system comprising a plurality of non-circular light sources and at least one camera, the method comprising:

emitting, via the plurality of non-circular light sources, light for illuminating an eye of the user when the head-mounted display apparatus in operation is worn by the user;

capturing, via the at least one camera, an image of the user's eye and reflections of the plurality of non-circular light sources from the user's eye;

identifying at least one of the plurality of non-circular light sources from where at least one of the reflections originated, based upon shapes, rotational orientations and relative positions of the reflections of the plurality of non-circular light sources, to differentiate said reflections from visual artifacts; and

detecting a gaze direction of the user based upon a relative position of a pupil of the user's eye with respect to the reflections of the plurality of non-circular light sources,

wherein at least two of the plurality of non-circular light sources have a same non-circular shape and the at least two of the plurality of non-circular light sources have different rotational orientations.

13. The method of claim 12 , wherein the plurality of non-circular light sources are arranged in a substantially circular manner, wherein the reflections of the plurality of non-circular light sources appear in the captured image as a plurality of non-circular glints arranged in a substantially circular manner.

14. The method of claim 13 , further comprising:

identifying a polygon defined by the plurality of non-circular glints; and

determining the relative position of the user's pupil with respect to the polygon.

15. The method of claim 13 , further comprising using the plurality of non-circular glints as references for a next image, to differentiate reflections of the plurality of non-circular light sources from visual artifacts in the next image.

16. The method of claim 12 , wherein a given non-circular light source has a shape that is symmetrical about only one axis and such shape only repeats when the light source is rotated through 360 degrees.

17. The method of claim 12 , further comprising:

dividing the plurality of non-circular light sources into a plurality of groups; and

controlling the plurality of groups to emit light in a predefined manner, based upon a time-division multiplexing rule.

Assignments (2)
SECURITY INTEREST Recorded Feb 13, 2020
From: VARJO TECHNOLOGIES OY
To: KREOS CAPITAL VI (UK) LIMITED
Reel/Frame 051930/0178 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 7, 2019
From: MIETTINEN, VILLE; MELAKARI, KLAUS
To: VARJO TECHNOLOGIES OY
Reel/Frame 047916/0955 →
Continuity (3)
Continuation In Part 15648886 · Jul 13, 2017
Continuation In Part 15366424 · Dec 1, 2016
Related Publication 20190138094A1 · May 9, 2019