IP Library Granted Patent US 9,069,382
Granted Patent B1
US 9,069,382 · App. 13/417,578 · Granted Jun 30, 2015

Using visual layers to aid in initiating a visual search

Inventors: Thad Eugene Starner (Mountain View, CA); Irfan Essa (Atlanta, GA)
Assignee: Google Inc.
G06F3/011G06F3/017G06F3/038
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,069,382
App. No.
13/417,578
Granted
Jun 30, 2015
Kind
B1
Abstract

Methods and devices for initiating a search are disclosed. In one embodiment, a method is disclosed that includes receiving video data captured by an image-capture device on a wearable computing device, segmenting the video data into a number of layers and, based on the video data, detecting that a pointing object is in proximity to a first layer. The method further includes initiating a first search on the first layer. In another embodiment, a wearable computing device is disclosed that includes an interface configured to receive video data captured by an image-capture device, a processor, and data storage comprising instructions executable by the processor to segment the video data into a number of layers and, based on the video data, detect that a pointing object is in proximity to a first layer. The instructions are further executable by the processor to initiate a first search on the first layer.

Claims (83)

1. A method comprising:

causing a camera on a wearable computing device to record video data;

segmenting the video data into a number of layers;

based on the video data, detecting that a pointing object is in proximity to a first layer;

initiating a first search on the first layer;

highlighting the first layer;

based on the video data, detecting that the pointing object is no longer in proximity to the first layer and is in proximity to a second layer; and

highlighting the second layer.

2. The method of claim 1 , further comprising causing a display to display first information corresponding to results of the first search.

3. The method of claim 2 , further comprising:

based on the video data, detecting that the pointing object is no longer in proximity to the first layer and is in proximity to a second layer;

initiating a second search on the second layer; and

replacing the first information with second information corresponding to results of the second search.

4. The method of claim 1 , wherein the layers exhibit at least one of motion coherence and appearance coherence.

5. The method of claim 1 , further comprising receiving an acceptance of the first layer.

6. The method of claim 5 , wherein receiving the acceptance comprises receiving at least one of a tactile input, an audible input, and a visual input.

7. The method of claim 1 , wherein the pointing device comprises a finger.

8. The method of claim 1 , wherein:

the camera is configured to be offset from an eye of a user by an angle and a distance; and

detecting that the pointing object is in proximity to the first layer is based on the angle and the distance.

9. The method of claim 1 , wherein segmenting the video data into the number of layers comprises:

segmenting the video data into a first set of layers in which the first layer of the first set of layers comprises both a first object and a second object;

based on the video data, detecting that the pointing object is behind the first object and in front of the second object; and

segmenting the video data into a second set of layers in which the first layer of the second set of layers comprises only one of the first object and the second object, wherein the number of layers comprise the second set of layers.

10. A wearable computing device comprising:

a camera configured to record video data;

a display;

at least one processor; and

data storage comprising instructions executable by the at least one processor to:

segment the video data into a number of layers;

based on the video data, detect that a pointing object is in proximity to a first layer;

initiate a first search on the first layer;

use the display to highlight the first layer;

based on the video data, detect that the pointing object is no longer in proximity to the first layer and is in proximity to a second layer; and

use the display to highlight the second layer.

11. The wearable computing device of claim 10 , wherein the display is further configured to display an indication of the layers.

12. The wearable computing device of claim 10 , wherein the display is further configured to display the video data.

13. The wearable computing device of claim 10 , wherein the display is further configured to display first information corresponding to results of the first search.

14. The wearable computing device of claim 10 , further comprising a user interface configured to receive an acceptance of the first layer.

15. The wearable computing device of claim 14 , wherein the user interface comprises at least one of a touchpad and a microphone.

16. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

causing a camera on a wearable computing device to record video data;

segmenting the video data into a number of layers;

based on the video data, detecting that a pointing object is in proximity to a first layer;

initiating a first search on the first layer;

highlighting the first layer;

based on the video data, detecting that the pointing object is no longer in proximity to the first layer and is in proximity to a second layer; and

highlighting the second layer.

17. The non-transitory computer-readable medium of claim 16 , wherein the functions further comprise displaying first information corresponding to results of the first search.

18. A method comprising:

causing a camera on a wearable computing device to record video data;

segmenting the video data into a first set of layers in which a first layer of the first set of layers comprises both a first object and a second object;

based on the video data, detecting that a pointing object is behind the first object and in front of the second object;

segmenting the video data into a second set of layers in which a first layer of the second set of layers comprises only one of the first object and the second object;

receiving an audible input;

initiating a visual search on one or more of the second set of layers;

based on the audible input, selecting information corresponding to at least some results of at least some of the visual searches; and

displaying the selected information.

19. The method of claim 18 , further comprising:

based on the video data, detecting that a pointing object is in proximity to the first layer of the second set of layers, and

wherein selecting information corresponding to at least some results of at least some of the visual searches comprises selecting the information based at least in part on a result of a visual search of the first layer of the second set of layers.

20. A method comprising:

causing a camera on a wearable computing device to record video data;

segmenting the video data into a number of layers;

based on the video data, detecting that a pointing object is in proximity to a first layer;

initiating a first search on the first layer;

causing a display to display first information corresponding to results of the first search;

based on the video data, detecting that the pointing object is no longer in proximity to the first layer and is in proximity to a second layer;

initiating a second search on the second layer; and

replacing the first information with second information corresponding to results of the second search.

21. The method of claim 20 , wherein:

the camera is configured to be offset from an eye of a user by an angle and a distance; and

detecting that the pointing object is in proximity to the first layer is based on the angle and the distance.

22. A method comprising:

causing a camera on a wearable computing device to record video data;

segmenting the video data into a first set of layers in which a first layer of the first set of layers comprises both a first object and a second object;

based on the video data, detecting that a pointing object is behind the first object and in front of the second object;

segmenting the video data into a second set of layers in which a first layer of the second set of layers comprises only one of the first object and the second object;

based on the video data, detecting that a pointing object is in proximity to the first layer of the second set of layers; and

initiating a first search on the first layer of the second set of layers.

23. The method of claim 22 , wherein:

the camera is configured to be offset from an eye of a user by an angle and a distance; and

detecting that the pointing object is in proximity to the first layer is based on the angle and the distance.

Assignments (3)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044334/0466 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2015
From: STARNER, THAD EUGENE
To: GOOGLE INC.
Reel/Frame 035476/0343 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 12, 2012
From: ESSA, IRFAN
To: GOOGLE INC.
Reel/Frame 027843/0667 →
Continuity (1)
Provisional Application 61584082 · Jan 6, 2012