IP Library Granted Patent US 9,405,977
Granted Patent B2
US 9,405,977 · App. 14/724,524 · Granted Aug 2, 2016

Using visual layers to aid in initiating a visual search

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,405,977
App. No.
14/724,524
Granted
Aug 2, 2016
Kind
B2
Abstract

Methods and devices for initiating a search are disclosed. In one embodiment, a method is disclosed that includes causing a camera on a wearable computing device to record video data, segmenting the video data into a number of layers and, based on the video data, detecting that a pointing object is in proximity to a first layer. The method further includes initiating a first search on the first layer. In another embodiment, a wearable computing device is disclosed that includes a camera configured to record video data, a processor, and data storage comprising instructions executable by the processor to segment the video data into a number of layers and, based on the video data, detect that a pointing object is in proximity to a first layer. The instructions are further executable by the processor to initiate a first search on the first layer.

Claims (58)

1. A method comprising:

causing a camera on a wearable computing device to record video data;

segmenting the video data into a number of layers that comprise at least a first layer and a second layer;

based on the video data, detecting a pointing object in a field of view of the camera;

determining that the pointing object is in proximity to the first layer, wherein the first layer corresponds to a first object;

in response to determining that the pointing object is in proximity to the first layer, initiating a visual search on at least a portion of the first layer; and

providing search results from the visual search to facilitate display of information related to the first object on a display that is associated with the wearable computing device.

2. The method of claim 1 , further comprising causing the display to display first information corresponding to the search results of the visual search.

3. The method of claim 2 , further comprising:

based on the video data, detecting that the pointing object is no longer in proximity to the first layer and is in proximity to the second layer;

initiating a second visual search on the second layer; and

replacing, on the display, the first information with second information corresponding to results of the second search.

4. The method of claim 1 , further comprising:

in further response to determining that the pointing object is in proximity to the first layer, highlighting the first layer.

5. The method of claim 4 , further comprising:

based on the video data, detecting that the pointing object is no longer in proximity to the first layer and is in proximity to the second layer; and

responsively highlighting the second layer.

6. The method of claim 1 , wherein each layer exhibits at least one of motion coherence and appearance coherence.

7. The method of claim 1 , further comprising receiving an acceptance of the first layer.

8. The method of claim 7 , wherein receiving the acceptance comprises receiving at least one of a tactile input, an audible input, and a visual input.

9. The method of claim 1 , wherein the pointing device comprises a finger.

10. The method of claim 1 , wherein the camera is configured to be offset from an eye of a user by an angle and a distance, and wherein the determining that the pointing object is in proximity to the first layer is based on the angle and the distance.

11. The method of claim 1 , wherein segmenting the video data into the number of layers comprises:

segmenting the video data into a first set of layers in which the first layer comprises both the first object and a second object;

based on the video data, determining that the pointing object is behind the first object and in front of the second object; and

segmenting the video data into a second set of layers in which the first layer comprises only one of the first object and the second object.

12. A wearable computing device comprising:

a camera interface operable to receive video data from a camera;

a display interface operable to output graphic data to a display;

at least one processor; and

data storage comprising instructions executable by the at least one processor to:

receive the video data via the camera interface;

segment the video data into a plurality of layers that comprise at least a first layer and a second layer;

based on the video data, detecting a pointing object in a field of view of the camera;

determine that the pointing object is in proximity to the first layer, wherein the first layer corresponds to a first object;

in response to determining that the pointing object is in proximity to the first layer, initiate a visual search on at least a portion of the first layer;

generate graphic data comprising information corresponding to search results from the visual search, wherein the information is related to the first object; and

output, via the display interface, the graphic data to the display.

13. The wearable computing device of claim 12 , further comprising a display configured to display an indication of the layers.

14. The wearable computing device of claim 13 , wherein the display is further configured to display the video data.

15. The wearable computing device of claim 13 , wherein the display is further configured to display first information corresponding to results of the first search.

16. The wearable computing device of claim 13 , wherein the display is further configured to highlight the first layer.

17. The wearable computing device of claim 12 , further comprising a user interface configured to receive an acceptance of the first layer.

18. The wearable computing device of claim 17 , wherein the user interface comprises at least one of a touchpad and a microphone.

19. A computing device comprising:

a communication interface operable to communicate with a wearable computing device;

at least one processor; and

data storage comprising instructions executable by the at least one processor to:

receive video data that was generated at the wearable computing device and is indicative of a field of view corresponding to the wearable computing;

segment the video data into a plurality of layers that comprise at least a first layer and a second layer;

based on the video data, detecting a pointing object in a field of view of the camera;

determine that the pointing object is in proximity to the first layer, wherein the first layer corresponds to a first object;

in response to determining that the pointing object is in proximity to the first layer, perform a visual search on at least a portion of the first layer to determine search results related to the first object; and

send at least a portion of the search results related to the wearable computing device, so as to facilitate display of information related to the first object on a display that is associated with the wearable computing device.

20. The computing device of claim 19 , wherein the segmentation of the video data into the number of layers comprises:

segmentation of the video data into a first set of layers in which the first layer comprises both the first object and a second object;

a determination, based on the video data, that the pointing object is behind the first object and in front of the second object; and

segmentation of the video data into a second set of layers in which the first layer comprises only one of the first object and the second object.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044566/0657 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2016
From: STARNER, THAD EUGENE; ESSA, IRFAN
To: GOOGLE INC.
Reel/Frame 038615/0354 →