IP Library Granted Patent US 9,536,354
Granted Patent B2
US 9,536,354 · App. 14/959,245 · Granted Jan 3, 2017

Object outlining to initiate a visual search

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,536,354
App. No.
14/959,245
Granted
Jan 3, 2017
Kind
B2
Abstract

Methods and devices for initiating a search of an object are disclosed. In one embodiment, a method is disclosed that includes receiving sensor data from a sensor on a wearable computing device and, based on the sensor data, detecting a movement that defines an outline of an area in the sensor data. The method further includes identifying an object that is located in the area and initiating a search on the object. In another embodiment, a server is disclosed that includes an interface configured to receive sensor data from a sensor on a wearable computing device, at least one processor, and data storage comprising instructions executable by the at least one processor to detect, based on the sensor data, a movement that defines an outline of an area in the sensor data, identify an object that is located in the area, and initiate a search on the object.

Claims (53)

1. A method comprising:

receiving video data from a camera on a wearable computing device, wherein a real-world field of view corresponds to the wearable computing device;

based on the video data, detecting a first movement of a pointing device in the real-world field of view;

determining that a real-world object is partially outlined by the first movement;

determining an identity of an object that is located in an area of the video data corresponding to the partial outline defined by the first movement of the pointing device;

initiating a search on the real-world object and outputting corresponding information related to the real-world object at the wearable computing device;

detecting, based on the video data, a second movement of a pointing device in the real-world field of view, wherein the second movement is a continuation of the first movement that extends the partial outline to provide a further-defined outline the real-world object and

responsive to detecting the second movement extending the outline of the real-world object, outputting additional information related to the real-world object at the wearable computing device.

2. The method of claim 1 , further comprising causing the wearable computing device to display the additional information.

3. The method of claim 1 , wherein initiating the search comprises:

beginning the search before the movement is complete.

4. The method of claim 3 , wherein:

beginning the search comprises causing the wearable computing device to provide information corresponding to results of the search.

5. The method of claim 1 , further comprising:

during the search, caching information corresponding to results of the search; and

causing the wearable computing device to provide at least some of the cached information.

6. The method of claim 1 , wherein the search comprises a visual search.

7. The method of claim 1 , wherein the pointing device comprises a finger.

8. The, method of claim 1 wherein:

the sensor is configured to be offset from an eye of a user by an angle and a distance; and

detecting the movement is further based on the angle and the distance.

9. The method of claim 1 , wherein determining the identity of the object comprises:

using edge detection to detect edges of the object; and

aligning the outline with at least one edge of the object.

10. The method of claim 1 , wherein determining the identity of the object comprises:

segmenting the sensor data into layers; and

aligning the outline with at least one layer.

11. The method of claim 1 , wherein outputting the information related to the real-world object at the wearable computing device comprises playing out audio.

12. The method of claim 1 , wherein outputting the additional information related to the real-world object at the wearable computing device comprises playing out audio.

13. A wearable computing device comprising:

an interface configured to receive video data from a camera on a wearable computing device, wherein a real-world field of view corresponds to the wearable computing device;

at least one processor; and

data storage comprising instructions executable by the at least one processor to:

based on the video data, detect a first movement of a pointing device in the real-world field of view;

determining that a real-world object is partially outlined by the first movement;

determine an identity of an object that is located in an area of the video data corresponding to the partial outline defined by the first movement of the pointing device;

initiate a search on the real-world object and outputting corresponding information related to the real-world object at the wearable computing device;

detect, based on the video data, a second movement of a pointing device in the real-world field of view, wherein the second movement is a continuation of the first movement that extends the partial outline to provide a further-defined outline the real-world object; and

responsive to detecting the second movement extending the outline of the real-world object, output additional information related to the real-world object at the wearable computing device.

14. The wearable computing device of claim 13 , further comprising a speaker configured to provide audio information corresponding to results of the search.

15. The wearable computing device of claim 14 , wherein the device comprises a head-mountable device.

16. The wearable computing device of claim 13 , wherein performing the search comprises:

beginning the search before the movement is complete.

17. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

receiving video data from a camera on a wearable computing device, wherein a real-world field of view corresponds to the wearable computing device;

based on the video data, detecting a first movement of a pointing device in the real-world field of view;

determining that a real-world object is partially outlined by the first movement;

determining an identity of an object that is located in an area of the video data corresponding to the partial outline defined by the first movement of the pointing device;

initiating a search on the real-world object and outputting corresponding information related to the real-world object at the wearable computing device;

detecting, based on the video data, a second movement of a pointing device in the real-world field of view, wherein the second movement is a continuation of the first movement that extends the partial outline to provide a further-defined outline the real-world object;

responsive to detecting the second movement extending the outline of the real-world object, outputting additional information related to the real-world object at the wearable computing device.

18. The non-transitory computer-readable medium of claim 17 , wherein performing the search comprises:

beginning the search before the movement is complete.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044097/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2015
From: STARNER, THAD EUGENE; ESSA, IRFAN
To: GOOGLE INC.
Reel/Frame 037222/0588 →