IP Library Granted Patent US 11,281,907
Granted Patent B2
US 11,281,907 · App. 15/494,114 · Granted Mar 22, 2022

Methods and systems for visual pairing of external devices with a wearable apparatus

Inventors: Yonatan Wexler (Jerusalem, IL); Amnon Shashua (Mevaseret Zion, IL)
Assignee: OrCam Technologies Ltd.
G06K9/00671G05B15/02G06F1/163G06F3/002G06F3/005G06F3/017G06F3/1454G06F3/16G06F3/167G06F16/22G06F16/51G06F16/583G06F16/5866G06F21/6245G06K7/10366G06K9/00335G06K9/00355G06K9/00389G06K9/00664G06K9/00684G06K9/00718G06K9/20G06K9/209G06K9/4604G06K9/6267G06Q10/103G08B21/18H04L65/4069H04N1/00342H04N5/2256H04N5/23203H04N5/23206H04N5/23222H04N5/23245H04N5/332H04N7/147H04N7/15H04N7/185H04W4/02H04W4/029G06Q30/0631
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,281,907
App. No.
15/494,114
Granted
Mar 22, 2022
Kind
B2
Abstract

A wearable apparatus is provided for capturing and processing images from an environment of a user. In one implementation, a wearable apparatus includes a transmitter, a memory, an image sensor configured to capture a stream of images from an environment of a user of the wearable apparatus, and a processing device. The processing device is programmed to receive the stream of images from the image sensor, analyze the stream of images to detect the external device in the environment of the user, and cause the transmitter to transmit an interrogation signal that is configured to cause a change in an aspect of the external device. The processing device is further programmed to analyze the stream of images to detect the change in the aspect of the external device and, after detection of the change, store in the memory information relating to the external device.

Claims (55)

1. A wearable apparatus for visually pairing with an external device, the wearable apparatus comprising:

at least one transmitter;

a memory;

at least one image sensor configured to capture a stream of images from an environment of a user of the wearable apparatus; and

at least one processing device programmed to:

receive the stream of images from the at least one image sensor;

analyze the stream of images to detect a plurality of external devices in the environment of the user, wherein detecting the plurality of external devices comprises:

analyzing the stream of images to identify an edge of at least one of the plurality of external devices; and

identifying the at least one of the plurality of external devices based on the identified edge;

cause the at least one transmitter to transmit a command to an intermediate device, the command being configured to cause the intermediate device to generate a subcommand and transmit the subcommand to one of the plurality of external devices, the subcommand comprising an interrogation signal being configured to cause a visually detectable change in at least one aspect of the one of the plurality of external devices, the visually detectable change including displaying information on a display of the one of the plurality of external devices for a predetermined period of time;

receive, using the at least one transmitter, an incoming signal confirmation from the intermediate device, the incoming signal comprising a confirmation that the interrogation signal was received by the one of the plurality of external devices;

after receiving the confirmation, analyze the stream of images to identify which of the detected plurality of external devices in the environment corresponds to the one of the plurality of external devices based on the information displayed on the display of the one of the plurality of external devices; and

based on the detection of the change in the at least one aspect of the external device, store in the memory information relating to the external device and generate feedback to the user of the wearable apparatus indicating the interrogation signal was received.

2. The wearable apparatus of claim 1 , wherein the information relating to the one of the plurality of external devices includes a location associated with the one of the plurality of external devices.

3. The wearable apparatus of claim 1 , wherein the information relating to the one of the plurality of external devices includes a type associated with the one of the plurality of external devices.

4. The wearable apparatus of claim 1 , wherein the information relating to the one of the plurality of external devices includes an indication of at least one controllable function associated with the one of the plurality of external devices.

5. The wearable apparatus of claim 1 , wherein the visually detectable change in the at least one aspect of the one of the plurality of external devices further includes a change in illumination of at least one light source associated with the one of the plurality of external devices.

6. The wearable apparatus of claim 5 , wherein the at least one light source includes an infrared light source.

7. The wearable apparatus of claim 5 , wherein the at least one light source includes a visible light source.

8. The wearable apparatus of claim 5 , wherein the change in illumination includes causing the at least one light source to at least one of: turn off, turn on, blink, dim, or brighten.

9. The wearable apparatus of claim 1 , wherein the at least one processing device is further programmed to:

analyze the stream of images to detect at least one recognized gesture made by the user; and

cause a control signal to be transmitted to the one of the plurality of external devices, the control signal being configured to control at least one aspect of the one of the plurality of external devices, the at least one aspect of the one of the plurality of external devices to be controlled being associated with the recognized gesture.

10. The wearable apparatus of claim 9 , wherein the control signal is configured to cause the one of the plurality of external devices to activate.

11. The wearable apparatus of claim 9 , wherein the control signal is configured to cause a change in illumination of a light source associated with the one of the plurality of external devices.

12. The wearable apparatus of claim 11 , wherein the change in illumination includes at least one of: turning on, turning off, dimming, or brightening.

13. The wearable apparatus of claim 1 , wherein analyzing the stream of images to detect the plurality of external devices in the environment of the user comprises:

detecting a group of external devices in the environment;

transmitting a subcommand to at least some of the group of external devices in the environment;

analyzing the stream of images to detect a visually detectable change in at least one aspect of the at least some of the group of external devices; and

differentiating among the group of external devices based on the detected change.

14. A method for visually pairing with an external device, the method comprising:

receiving a stream of images captured from an environment of a user of a wearable apparatus;

analyzing the stream of images to detect a plurality of external devices in the environment of the user, wherein detecting the plurality of external devices comprises:

analyzing the stream of images to identify an edge of at least one of the plurality of external devices; and

identifying the at least one of the plurality of external devices based on the identified edge;

causing at least one transmitter associated with the wearable apparatus to transmit an interrogation signal a command to an intermediate device, the command being configured to cause the intermediate device to generate a subcommand and transmit the subcommand to one of the plurality of external devices, the subcommand comprising an interrogation signal being configured to cause a visually detectable change in at least one aspect of the one of the plurality of external devices;

receiving, using the at least one transmitter, an incoming signal confirmation from the intermediate device, the incoming signal comprising a confirmation that the interrogation signal was received by the one of the plurality of external devices;

after receiving the confirmation, analyzing the stream of images to identify which of the detected plurality of external devices in the environment based on detecting the visually detectable change in the at least one aspect of the one of the plurality of external devices, the visually detectable change including displaying information on a display of the one of the plurality of external devices for a predetermined period of time; and

based on the information displayed on the display of the external device, storing information relating to the external device and generating feedback to the user of the wearable apparatus indicating the interrogation signal was received.

15. The method of claim 14 , wherein the information relating to the one of the plurality of external devices includes a location associated with the one of the plurality of external devices.

16. The method of claim 14 , wherein the information relating to the one of the plurality of external devices includes a type associated with the one of the plurality of external devices.

17. The method of claim 14 , wherein the information relating to the one of the plurality of external devices includes an indication of at least one controllable function associated with the external device one of the plurality of external devices.

18. The method of claim 14 , wherein the visually detectable change in the at least one aspect of the one of the plurality of external devices further includes a change in illumination of at least one light source associated with the one of the plurality of external devices.

19. The method of claim 14 , further comprising:

analyzing the stream of images to detect at least one recognized gesture made by the user; and

causing a control signal to be transmitted to the one of the plurality of external devices, the control signal being configured to control at least one aspect of the one of the plurality of external devices, the at least one aspect of the one of the plurality of external devices to be controlled being associated with the recognized gesture.

20. The method of claim 19 , wherein the control signal is configured to cause the one of the plurality of external devices to activate.

21. The method of claim 19 , wherein the control signal is configured to cause a change in illumination of a light source associated with the one of the plurality of external devices.

22. The method of claim 14 , wherein analyzing the stream of images to detect the one of the plurality of external devices in the environment of the user comprises:

detecting a group of external devices in the environment;

transmitting a subcommand to at least some of the group of external devices in the environment;

analyzing the stream of images to detect a visually detectable change in at least one aspect of the at least some of the group of external devices; and

differentiating among the group of external devices based on the detected change.

23. A non-transitory computer readable medium storing computer implementable instructions for carrying out the method of claim 14 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2017
From: WEXLER, YONATAN; SHASHUA, AMNON
To: ORCAM TECHNOLOGIES LTD.
Reel/Frame 042138/0351 →
Continuity (3)
Continuation 15399612 · Jan 5, 2017
Provisional Application 62275531 · Jan 6, 2016
Related Publication 20170228029A1 · Aug 10, 2017