IP Library › Granted Patent US 10,845,871
Granted Patent B2
US 10,845,871 · App. 16/056,740 · Granted Nov 24, 2020

Interaction and management of devices using gaze detection

Inventors: Crystal Lee Parker (Duvall, WA); Mark Louis Wilson O'Hanlon (Woodinville, WA); Andrew Lovitt (Redmond, WA); Jason Ryan Farmer (Snohomish, WA)
Assignee: Microsoft Technology Licensing, LLC
G06F3/013G06F3/0482G06F3/04842G06F3/165G06F3/167G10L15/22G06F2203/0381G10L2015/228
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,845,871
App. No.
16/056,740
Granted
Nov 24, 2020
Kind
B2
Abstract

A controller is adapted to recognize an input from a user using an input interface, determine if a user gaze information indicates that the user is gazing at a device, and when the user gaze information indicates that the user is gazing at the device, route response information to the device.

Claims (46)

1. A controller adapted to:

recognize an audio input from a user using an input interface;

determine if a user gaze information from a gaze detector indicates that the user is gazing at a device of a plurality of devices; and

when the user gaze information from the gaze detector indicates that the user is gazing at a first device:

based on the user gaze information, activate a first interaction set of input commands recognizable by the controller and associated with the first device from a plurality of interaction sets, the plurality of interaction sets being associated with the plurality of devices, and

in response to the audio input from the input interface route response information to the first device, and

when the user gaze information from the gaze detector indicates that the user is gazing at a second device:

based on the user gaze information, activate a second interaction set of input commands recognizable by the controller and associated with the second device from the plurality of interaction sets, and

in response to the audio input from the input interface route response information to the second device.

2. The controller of claim 1 further adapted to:

when user gaze information does not indicate that the user is gazing at the plurality of devices, provide a response to the input.

3. The controller of claim 2 , the response to the input is an audio response.

4. The controller of claim 1 , the audio input being a speech input, where the controller is further adapted to:

based on the user gaze information indicating that the user is gazing at the device, activate an interaction set of audio commands recognizable by the controller; and

respond, via the interaction set of audio commands, with a command corresponding to the speech input of the user.

5. The controller of claim 1 , the audio input being a speech input, the speech input ambiguously matching a plurality of audio commands and the user gaze information indicating that the user is gazing at the device, the controller further adapted to:

based on the gaze information indicating that the user is gazing at the device, select an audio command from the plurality of audio commands based on the device.

6. The controller of claim 1 further adapted to receive gaze information from a plurality of users and speech inputs from the plurality of users.

7. The controller of claim 6 further adapted to activate a first interaction set based on a first gaze information for a first user and a second interaction set based on a second gaze information for a second user.

8. The controller of claim 7 further adapted to correlate a first audio command from the first user to the first gaze information and correlate a second audio command from the second user to the second gaze information.

9. A method, comprising:

receiving, by a gaze detector, a user gaze information;

receiving, by an input interface, an audio input from a user;

selecting at least one interaction set of a plurality of interaction sets recognizable by a controller based on the user gaze information received from the gaze detector; and

responding to the audio input from the input interface with a response based on the interaction set and the user gaze information.

10. The method of claim 9 further comprising selecting at least one input detector of the input interface from a plurality of input detectors based on the user gaze information.

11. The method of claim 9 , the user gaze information including at least one of a user line of sight, a user point of focus, or an area that the user is not viewing.

12. The method of claim 9 , the user gaze information identifying a device at which the user is looking.

13. The method of claim 12 , the user gaze information further identifying a region of the device at which the user is looking.

14. The method of claim 9 , the response including an audio response.

15. The method of claim 9 , the response including a haptic response.

16. The method of claim 9 , the user gaze information including a gaze duration.

17. The method of claim 9 , selecting at least one an interaction set based on the user gaze information further comprising deselecting at least a second interaction set.

18. The method of claim 9 , the audio input being a speech input.

19. The method of claim 9 , the audio input further including a gesture input.

20. A system comprising:

an input interface, the input interface providing an audio input; and

a processor in data communication with the input interface, a gaze detector, and a storage device, the storage device having instructions stored thereon, which when executed by the processor, cause the processor to:

based on gaze information indicating that a user is gazing at a first device of a plurality of devices:

select a first interaction set corresponding to the first device,

process the audio input based on the first interaction set, and

route response information to the first device; and

based on gaze information indicating that a user is gazing at second first device of the plurality of devices:

select a second interaction set corresponding to the second device,

process the audio input based on the second interaction set, and

route response information to the second device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2018
From: PARKER, CRYSTAL LEE; O'HANLON, MARK LOUIS WILSON; LOVITT, ANDREW; FARMER, JASON RYAN
To: MICROSOFT CORPORATION
Reel/Frame 046666/0587 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2018
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 046666/0628 →
Continuity (3)
Continuation 15796295 · Oct 27, 2017
Continuation 13474723 · May 18, 2012
Related Publication 20180341330A1 · Nov 29, 2018
Cited By (2)
US 12,210,677 US 12,688,053