IP Library Granted Patent US 9,563,272
Granted Patent B2
US 9,563,272 · App. 15/132,170 · Granted Feb 7, 2017

Gaze assisted object recognition

Inventors: Kenneth M. Karakotsios (San Jose, CA); Isaac S. Noble (Soquel, CA)
Assignee: Amazon Technologies, Inc.
G06F3/013G06F3/012G06K9/00201G06K9/00221G06K9/00597G06T7/00H04N5/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,563,272
App. No.
15/132,170
Granted
Feb 7, 2017
Kind
B2
Abstract

An electronic device can attempt to determine a gaze direction of a user, which can be used to determine an object of interest to the user. Determining the gaze direction helps to reduce the search space and reduce processing requirements for identifying the object. Image information can be captured that includes the object of interest, which then can be analyzed to recognize the type of object. Upon recognizing the object, the user can be provided with information about the object, which in some cases can depend at least in part upon a current context or location of the object. If the object is a networked device, the user can potentially be provided with input options for controlling the device.

Claims (46)

1. A computer-implemented method, comprising:

under control of one or more computing systems configured with executable instructions, receiving image data from a camera;

determining, from the image data and using at least one processor of a computing device, a relative position of a user in a first portion of a field of view of the camera;

determining, from the image data, that an object is in a second portion of the field of view;

determining, based at least in part on the relative position of the user, that a face of the user is directed toward the object;

recognizing, based at least in part on the image data, a characteristic of the object;

obtaining first information associated with the object;

providing second information, the second information related to the first information;

receiving, via a user command, a selection associated with the second information; and

causing the object to be operated.

2. The computer-implemented method of claim 1 , further comprising connecting the computing device to the object by at least a wireless electronic control channel.

3. The computer-implemented method of claim 1 , further comprising connecting the computing device to the object by at least a wired electronic control channel.

4. The computer-implemented method of claim 1 , further comprising receiving the image data from at least one of a video camera, a digital camera, an infrared-capable camera or a stereoscopic camera.

5. The computer-implemented method of claim 1 , further comprising providing the second information using at least a speaker.

6. The computer-implemented method of claim 1 , further comprising receiving the selection associated with the second information using at least a microphone.

7. The computer-implemented method of claim 1 , further comprising:

receiving the image data from a video camera, the image data including at least a portion of a face of the user;

locating position information for the user's eyes in the image data; and

determining that the face of the user is directed toward the object based at least in part on the position information for the user's eyes.

8. The computer-implemented method of claim 1 , further comprising:

identifying the object using at least one of an image recognition algorithm, a pattern recognition algorithm, a barcode detection algorithm, or an optical character recognition algorithm.

9. The computer-implemented method of claim 1 , further comprising receiving the selection associated with the second information by at least one of a motion, a gesture, an audio command, or a physical input.

10. The computer-implemented method of claim 1 , further comprising providing at least a portion of the image data to a service configured to recognize the characteristic of the object.

11. A computing system, comprising:

a camera;

at least one processor; and

a memory device including instructions that, when executed by the at least one processor, cause the computing system to:

receive image data from the camera;

determine, from the image data and using the at least one processor, a relative position of a user in a first portion of a field of view of the camera;

determine, from the image data, that an object is in a second portion of the field of view;

determine, based at least in part on the relative position of the user, that a face of the user is directed toward the object;

recognize, based at least in part on the image data, a characteristic of the object;

obtain first information associated with the object;

provide second information, the second information related to the first information;

receive, via a user input, a selection associated with the second information; and

causing the object to be operated.

12. The computing system according to claim 11 , further comprising an electronic communication channel connecting the computing system to the object.

13. The computing system according to claim 12 , wherein the electronic communication channel is a wireless electronic control channel.

14. The computing system according to claim 12 , wherein the electronic communication channel is a wired electronic control channel.

15. The computing system according to claim 12 , wherein the instructions that, when executed by the at least one processor, further cause the computing system to provide a control scheme via the electronic communication channel enabling the computing system to function as a control mechanism for the object.

16. The computing system according to claim 11 , wherein the camera includes at least one of a video camera, a digital camera, an infrared-capable camera and a stereoscopic camera.

17. The computing system according to claim 16 , wherein the instructions that, when executed by the at least one processor, further cause the computing system to receive the image data from the at least one of a video camera, a digital camera, an infrared-capable camera or a stereoscopic camera.

18. The computing system according to claim 11 , further comprising:

at least one audio component, including at least one of a speaker and a microphone.

19. The computing system according to claim 18 , wherein the instructions that, when executed by the at least one processor, further cause the computing system to provide the second information using at least the speaker.

20. The computing system according to claim 18 , wherein the instructions that, when executed by the at least one processor, further cause the computing system to receive the selection using at least the microphone.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2016
From: KARAKOTSIOS, KENNETH M.; NOBLE, ISAAC S.
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 039126/0993 →
Continuity (2)
Continuation 13485195 · May 31, 2012
Related Publication 20160283778A1 · Sep 29, 2016