IP Library Granted Patent US 9,244,539
Granted Patent B2
US 9,244,539 · App. 14/149,756 · Granted Jan 26, 2016

Target positioning with gaze tracking

Inventors: Morgan Kolya Venable (San Francisco, CA); Bernard James Kerr (Sausalito, CA); Weerapan Wilairat (Sammamish, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06F3/0346A61B3/113G02B27/017G02B27/0172G06F3/012G06F3/013G06F3/038G06F3/04842G06K9/00671G02B2027/014G02B2027/0156G02B2027/0174G02B2027/0178G02B2027/0187G06F2201/805G06F2209/508
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,244,539
App. No.
14/149,756
Granted
Jan 26, 2016
Kind
B2
Abstract

Embodiments that relate to positioning a target indicator via a display system are disclosed. For example, one disclosed embodiment provides a method for positioning a target indicator using gaze tracking data having a coarse accuracy from a gaze tracking system of a computing device. Head pose data having a fine accuracy greater than the coarse accuracy is received from a head tracking system. Using the gaze tracking data, an approximate user gaze region within a display region is determined, and the target indicator is displayed at an initial location within the approximate user gaze region. A reposition input from the user is received. In response, subsequently received head pose data is used to calculate an adjusted location for the target indicator. The target indicator is then displayed at the adjusted location.

Claims (57)

1. A head-mounted computing device for positioning a target indicator displayed via a display system, the head-mounted computing device operatively connected to the display system, the head-mounted computing device comprising:

a gaze tracking system for generating gaze tracking data having a coarse accuracy from a user wearing the head-mounted computing device;

a head tracking system for generating head pose data from the user, wherein the head pose data has a fine accuracy that is greater than the coarse accuracy of the gaze tracking data; and

a positioning module executed by a processor of the head-mounted computing device, the positioning module configured to:

using the gaze tracking data, determine an approximate user gaze region within a display region generated by the display system;

control the display system to display the target indicator at an initial location within the approximate user gaze region;

receive a reposition input from the user;

in response to receiving the reposition input, use subsequently received head pose data to calculate an adjusted location for the target indicator in the display region; and

control the display system to display the target indicator at the adjusted location in the display region.

2. The head-mounted computing device of claim 1 , wherein the positioning module is further configured to:

after the target indicator is displayed at the adjusted location, receive a selection user input; and

in response to receiving the selection user input, select a selectable target located at the adjusted location.

3. The head-mounted computing device of claim 1 , wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

4. The head-mounted computing device of claim 1 , wherein the target indicator is not displayed until the reposition input is received.

5. The head-mounted computing device of claim 1 , wherein the positioning module is further configured to, in response to receiving the reposition input, disable the gaze tracking system.

6. The head-mounted computing device of claim 1 , wherein the display system comprises a near-eye display system integrated in the head-mounted computing device.

7. The head-mounted computing device of claim 1 , wherein the display system comprises a screen in a physical environment and separate from the head-mounted computing device, the head-mounted computing device further comprises an optical sensor for capturing images of the physical environment including the screen, and the positioning module is further configured to:

use the images to establish a spatial relationship between the head-mounted computing device and the screen; and

use the spatial relationship in combination with the gaze tracking data to determine the approximate user gaze region within the display region of the screen.

8. The head-mounted computing device of claim 7 , wherein the positioning module is further configured to receive a screen identifier from the screen.

9. The head-mounted computing device of claim 1 , wherein the positioning module is further configured to:

receive secondary tracking data; and

in response to receiving the reposition input, use subsequently received secondary tracking data to calculate the adjusted location for the target indicator.

10. A method for positioning a target indicator displayed via a display system, the method comprising:

receiving gaze tracking data having a coarse accuracy from a gaze tracking system of a computing device;

receiving head pose data having a fine accuracy that is greater than the coarse accuracy of the gaze tracking data, the head pose data received from a head tracking system of the computing device;

using the gaze tracking data, determining an approximate user gaze region within a display region generated by the display system;

controlling the display system to display the target indicator at an initial location within the approximate user gaze region;

receiving a reposition input from the user;

in response to receiving the reposition input, using subsequently received head pose data to calculate an adjusted location for the target indicator in the display region; and

controlling the display system to display the target indicator at the adjusted location in the display region.

11. The method of claim 10 , further comprising:

after the target indicator is displayed at the adjusted location, receiving a selection user input; and

in response to receiving the selection user input, selecting a selectable target located at the adjusted location.

12. The method of claim 10 , wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

13. The method of claim 10 , further comprising not displaying the target indicator until the reposition input is received.

14. The method of claim 10 , further comprising disabling the gaze tracking system in response to receiving the reposition input.

15. The method of claim 10 , wherein the display system comprises a screen integrated in the computing device.

16. The method of claim 10 , wherein the display system comprises a screen in a physical environment and separate from the computing device, the computing device further comprises an optical sensor for capturing images of the physical environment including the screen, and the method further comprising:

using the images to establish a spatial relationship between the computing device and the screen; and

using the spatial relationship in combination with the gaze tracking data to determine the approximate user gaze region within the display region of the screen.

17. The method of claim 16 , further comprising receiving a screen identifier from the screen.

18. The method of claim 10 , further comprising:

receiving secondary tracking data; and

in response to receiving the reposition input, using the secondary tracking data to calculate the adjusted location for the target indicator.

19. A head-mounted display device for positioning a target indicator displayed via a display system, the head-mounted display device operatively connected to a computing device, the head-mounted display device comprising:

a gaze tracking system for generating gaze tracking data having a coarse accuracy from a user wearing the head-mounted computing device;

a head tracking system for generating head pose data from the user, wherein the head pose data has a fine accuracy that is greater than the coarse accuracy of the gaze tracking data; and

a positioning module executed by a processor of the computing device, the positioning module configured to:

using the gaze tracking data, determine an approximate user gaze region within a display region generated by the display system;

receive a reposition input from the user;

in response to receiving the reposition input, control the display system to display the target indicator at an initial location within the approximate user gaze region;

in response to receiving the reposition input, use subsequently received head pose data to calculate an adjusted location for the target indicator in the display region;

control the display system to display the target indicator at the adjusted location in the display region;

after the target indicator is displayed at the adjusted location, receive a selection user input; and

in response to receiving the selection user input, select a selectable target located at the adjusted location.

20. The head-mounted display device of claim 19 , wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2015
From: VENABLE, MORGAN KOLYA; KERR, BERNARD JAMES; WILAIRAT, WEERAPAN
To: MICROSOFT CORPORATION
Reel/Frame 036728/0859 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
Continuity (1)
Related Publication 20150193018A1 · Jul 9, 2015