IP Library Granted Patent US 10,380,425
Granted Patent B2
US 10,380,425 · App. 15/495,650 · Granted Aug 13, 2019

Wearable apparatus and methods for processing images to identify contextual situations

Inventors: Yonatan Wexler (Jerusalem, IL); Amnon Shashua (Mevaseret Zion, IL)
Assignee: OrCam Technologies Ltd.
G06K9/00671G05B15/02G06F1/163G06F3/002G06F3/005G06F3/017G06F3/1454G06F3/16G06F3/167G06F16/22G06F16/51G06F16/583G06F16/5866G06F21/6245G06K7/10366G06K9/00335G06K9/00355G06K9/00389G06K9/00664G06K9/00684G06K9/00718G06K9/20G06K9/209G06K9/4604G06K9/6267G06Q10/103G08B21/18H04L65/4069H04N1/00342H04N5/2256H04N5/23203H04N5/23206H04N5/23222H04N5/23245H04N5/332H04N7/147H04N7/15H04N7/185H04W4/02G06Q30/0631
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,380,425
App. No.
15/495,650
Granted
Aug 13, 2019
Kind
B2
Abstract

A wearable apparatus is provided for capturing and processing images from an environment of a user. In one implementation, the wearable apparatus is used for identifying a contextual situation related to a wearer. The wearable apparatus includes a wearable image sensor configured to capture a plurality of images from an environment of the wearer. The wearable apparatus further includes at least one processing device. The at least one processing device is programmed to analyze the plurality of images to identify the contextual situation related to the wearer; determine information associated with the contextual situation; and cause the transmitter to transmit the determined information to a device paired with the wearable apparatus to cause the paired device to provide at least one alert to the wearer based on the determined information associated with the contextual situation.

Claims (57)

1. A wearable apparatus for identifying contextual situations related to a user wearing the wearable apparatus, the wearable apparatus comprising:

a wearable image sensor configured to capture a plurality of images from an environment of the user wearing the wearable apparatus;

a transmitter; and

at least one processing device programmed to:

analyze the plurality of images to identify a first contextual situation related to the user wearing the wearable apparatus;

use the plurality of images to determine information associated with the first contextual situation related to the user wearing the wearable apparatus;

use at least of one image-information from previously obtained images and additional information from one or more sources to improve a certainty level in the identification of the first contextual situation;

cause the transmitter to transmit the determined information to a device paired with the wearable apparatus through a short-range communication link to cause the paired device to provide at least one alert to the user wearing the wearable apparatus based on the determined information associated with the first contextual situation identified in the plurality of images;

analyze at least one image captured by the wearable image sensor to identify a second contextual situation related to the user wearing the wearable apparatus;

determine a time difference between the first contextual situation and the second contextual situation; and

based on the first contextual situation, image-information associated with the second contextual situation, and a comparison of the determined time difference with a predetermined threshold associated with the first contextual situation, withhold transmission to the paired device of alerts associated with the second contextual situation.

2. The wearable apparatus of claim 1 , wherein the first contextual situation identified in the plurality of images includes the user wearing the wearable apparatus transitioning from indoors to outdoors.

3. The wearable apparatus of claim 2 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes information related to a weather condition.

4. The wearable apparatus of claim 2 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes a suggestion to remember rain gear.

5. The wearable apparatus of claim 2 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes a suggestion to remember a key.

6. The wearable apparatus of claim 2 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes a suggestion to change an HVAC setting.

7. The wearable apparatus of claim 1 , wherein the first contextual situation identified in the plurality of images includes the user wearing the wearable apparatus entering a grocery store.

8. The wearable apparatus of claim 7 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes a suggestion to purchase an item.

9. The wearable apparatus of claim 8 , wherein the suggestion to purchase an item is based on a prior determination by the at least one processing device, based on prior captured images, that a container associated with the item was discarded by the user wearing the wearable apparatus.

10. The wearable apparatus of claim 1 , wherein the first contextual situation identified in the plurality of images includes the user wearing the wearable apparatus exiting a vehicle.

11. The wearable apparatus of claim 10 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes a reminder indicating that a child is present in the vehicle.

12. The wearable apparatus of claim 1 , wherein the first contextual situation identified in the plurality of images includes an identification of at least one worker at a work site and the at least one alert triggered in the paired device in response to the transmission of the determined information indicates that the worker is not using and/or wearing safety equipment.

13. The wearable apparatus of claim 1 , wherein the paired device is at least one of: a smartphone, a smarthome controller, a tablet, or a smart watch that is located in proximity to the wearable apparatus.

14. The wearable apparatus of claim 1 , wherein the first contextual situation identified in the plurality of images includes a document present in an area in front of the user wearing the wearable apparatus.

15. The wearable apparatus of claim 14 , wherein the determined information transmitted from the wearable apparatus through the short-range communication link to the paired device includes an address appearing on the document.

16. The wearable apparatus of claim 15 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes the address appearing in a user interface associated with a navigation assistance application.

17. The wearable apparatus of claim 14 , wherein the determined information transmitted from the wearable apparatus through the short-range communication link to the paired device includes text from a business card.

18. The wearable apparatus of claim 17 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes the text from the business card appearing in a user interface associated with an application on the paired device.

19. The wearable apparatus of claim 14 , wherein the determined information transmitted from the wearable apparatus through the short-range communication link to the paired device includes text from a financial document.

20. The wearable apparatus of claim 19 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information includes text from the financial document appearing in a user interface associated with a financial-related application on the paired device.

21. The wearable apparatus of claim 1 , wherein the at least one alert triggered in the paired device in response to the transmission of the determined information is associated with at least one of a determined reading speed of the user, a last page read of a book, or progress toward a predetermined reading goal.

22. The wearable apparatus of claim 1 , wherein the at least one processing device is further programmed to:

execute at least one machine learning algorithm to determine a likelihood that providing the at least one alert will cause the user wearing the wearable apparatus to perform an action.

23. The wearable apparatus of claim 1 , wherein the at least one processing device is further programmed to:

communicate with multiple paired devices; and

select one of the multiple paired devices based on the distance of each paired device from the wearable apparatus.

24. The wearable apparatus of claim 1 , wherein causing the paired device to provide at least one alert to the user wearing the wearable apparatus includes causing a display of the paired device to visibly output information.

25. A method for identifying contextual situations related to a user wearing a wearable apparatus, the method comprising:

receiving a plurality of images captured from an environment of the user wearing the wearable apparatus;

analyzing the plurality of images to identify a first contextual situation related to the user wearing the wearable apparatus;

using the plurality of images to determine information associated with the first contextual situation related to the user wearing the wearable apparatus;

causing a device paired with the wearable apparatus through a short-range communication link to provide at least one alert to the user wearing the wearable apparatus based on the determined information associated with the first contextual situation identified in the plurality of images;

after causing the device paired with the wearable apparatus to provide the at least one alert, receiving at least one image captured from an environment of the user wearing the wearable apparatus;

analyzing the at least one image to identify a second contextual situation related to the user wearing the wearable apparatus;

determining a time difference between the first contextual situation and the second contextual situation; and

based on the first contextual situation, image-information associated with the second contextual situation, and a comparison of the determined time difference with a predetermined threshold associated with the first contextual situation, withhold transmission of information that triggers alerts associated with the second contextual situation.

26. The method of claim 25 , wherein the second contextual situation is repetitive of the first contextual situation.

27. The method of claim 25 , wherein the second contextual situation involves objects associated with the first contextual situation.

28. A non-transitory computer readable medium and comprising data and computer implementable instructions for carrying out a method for identifying contextual situations related to a user wearing a wearable apparatus, the method comprising:

receiving a plurality of images captured from an environment of the user wearing the wearable apparatus;

analyzing the plurality of images to identify a first contextual situation related to the user wearing the wearable apparatus;

using the plurality of images to determine information associated with the first contextual situation related to the user wearing the wearable apparatus;

causing a device paired with the wearable apparatus through a short-range communication link to provide at least one alert to the user wearing the wearable apparatus based on the determined information associated with the first contextual situation identified in the plurality of images;

after causing the device paired with the wearable apparatus to provide the at least one alert, receiving at least one image captured from an environment of the user wearing the wearable apparatus;

analyzing the at least one image to identify a second contextual situation related to the user wearing the wearable apparatus;

determining a time difference between the first contextual situation and the second contextual situation; and

based on the first contextual situation, image-information associated with the second contextual situation, and a comparison of the determined time difference with a predetermined threshold associated with the first contextual situation, withhold transmission of information that triggers alerts associated with the second contextual situation.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 3, 2017
From: WEXLER, YONATAN; SHASHUA, AMNON
To: ORCAM TECHNOLOGIES LTD.
Reel/Frame 042391/0006 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2017
From: WEXLER, YONATAN
To: ORCAM TECHNOLOGIES LTD.
Reel/Frame 042134/0606 →
Continuity (3)
Continuation 15399310 · Jan 5, 2017
Provisional Application 62275531 · Jan 6, 2016
Related Publication 20170228596A1 · Aug 10, 2017