IP Library Granted Patent US 11,638,103
Granted Patent B2
US 11,638,103 · App. 17/284,700 · Granted Apr 25, 2023

Identifying information and associated individuals

Inventors: Tal Rosenwein (Jerusalem, IL); Yonatan Wexler (Jerusalem, IL); Amnon Shashua (Mevaseret Zion, IL); Roi Nathan (Jerusalem, IL)
Assignee: Orcam Technologies Ltd.
H04R25/405G03B31/00G06F1/163G06F1/1686G06F3/165G06F3/167G06F18/21G06F18/251G06V10/803G06V20/10G06V40/10G06V40/16G06V40/165G06V40/171G06V40/172G06V40/20G10L15/26G10L17/00G10L17/04G10L17/06G10L17/18G10L21/003G10L21/0272G10L21/034G10L25/51H04N5/38H04N7/185H04N23/51H04R1/08H04R25/407H04R25/45H04R25/505H04R25/554H04R25/558H04R25/60H04R25/606H04R25/65G06K9/00H04R2225/025H04R2225/41H04R2225/43H04R2225/55H04R2460/01H04R2460/13
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,638,103
App. No.
17/284,700
Granted
Apr 25, 2023
Kind
B2
Abstract

A hearing aid system for individual identification of a hearing aid system may include a wearable camera, a microphone, and at least one processor. The processor may be programmed to receive a plurality of images captured by the wearable camera; receive audio signals representative of sounds captured by the microphone; and identify a first audio signal, from among the received audio signals, representative of a voice of a first individual. The processor may transcribe and store, in a memory, text corresponding to speech associated with the voice of the first individual and determine whether the first individual is a recognized individual. If the first individual is a recognized individual, the processor may associate an identifier of the first recognized individual with the stored text corresponding to the speech associated with the voice of the first individual.

Claims (60)

1. A hearing aid system, comprising:

at least one microphone configured to capture sounds from an environment of a user; and

at least one processor programmed to:

receive audio signals representative of sounds captured by the at least one microphone;

identify a first audio signal, from among the received audio signals, representative of a voice of a first individual;

transcribe and store, in a memory, text corresponding to speech associated with the voice of the first individual;

determine whether the first individual is a recognized individual; and

if the first individual is a recognized individual, associate an identifier of the first recognized individual with the stored text corresponding to the speech associated with the voice of the first individual.

2. The hearing aid system of claim 1 , wherein the at least one processor is further programmed to prompt the user for identification information relating to the first individual if the first individual is not a recognized individual.

3. The hearing aid system of claim 1 , wherein the at least one processor is further programmed to determine whether the speech associated with the voice of the first individual is directed toward the user.

4. The hearing aid system of claim 3 , wherein the at least one processor is further programmed to store in the memory an indication that the speech associated with the voice of the first individual is directed toward the user if the speech associated with the voice of the first individual is directed toward the user.

5. The hearing aid system of claim 3 , wherein determination of whether the speech associated with the voice of the first individual is directed toward the user is based on at least one of a detected look direction of the user or a detected look direction of the first individual.

6. The hearing aid system of claim 5 , wherein the hearing aid system further comprises a wearable camera configured to capture a plurality of images from the environment of the user, wherein the at least one processor is further configured to receive the plurality of images captured by the wearable camera, and wherein the detected look direction of the user is determined based on detection of a chin of the user in at least one of the plurality of images.

7. The hearing aid system of claim 5 , wherein the hearing aid system further comprises a wearable camera configured to capture a plurality of images from the environment of the user, wherein the at least one processor is further configured to receive the plurality of images captured by the wearable camera, and wherein the detected look direction of the first individual is determined based on detection of one or more eyes of the first individual in at least one of the plurality of images and based on at least one characteristic of the one or more detected eyes.

8. The hearing aid system of claim 1 , wherein the hearing aid system further comprises a wearable camera configured to capture a plurality of images from the environment of the user, and wherein the at least one processor is further programmed to:

receive the plurality of images captured by the wearable camera;

identify a second individual represented in at least one of the plurality of images;

identify a second audio signal, from among the received audio signals, representative of a voice of the second individual; and

transcribe and store, in the memory, text corresponding to speech associated with the voice of the second individual.

9. The hearing aid system of claim 8 , wherein the at least one processor is further programmed to:

determine whether the second individual is a recognized individual; and

if the second individual is a recognized individual, associate an identifier of the second recognized individual with the stored text corresponding to the speech associated with the voice of the second individual.

10. The hearing aid system of claim 8 , wherein the at least one processor is further programmed to:

determine whether the speech associated with the voice of the second individual is directed toward the user; and

if the speech associated with the voice of the second individual is directed toward the user, store in the memory an indication that the speech associated with the voice of the second individual is directed toward the user.

11. The hearing aid system of claim 8 , wherein the at least one processor is further programmed to:

determine whether the speech associated with the voice of the second individual is directed toward the first individual; and

if the speech associated with the voice of the second individual is directed toward the first individual, store in the memory an indication that the speech associated with the voice of the second individual is directed toward the first individual.

12. The hearing aid system of claim 11 , wherein determination of whether the speech associated with the voice of the second individual is directed toward the first individual is based on a look direction of the second individual detected based on analysis of at least one of the plurality of images.

13. The hearing aid system of claim 11 , wherein determination of whether the speech associated with the voice of the second individual is directed toward the first individual is based on detection, in the speech of the second individual, a name associated with the first individual.

14. The hearing aid system of claim 8 , wherein the at least one processor is further programmed to:

determine whether the speech associated with the voice of the first individual is directed toward the second individual; and

if the speech associated with the voice of the first individual is directed toward the second individual, store in the memory an indication that the speech associated with the voice of the first individual is directed toward the second individual.

15. The hearing aid system of claim 8 , wherein the at least one processor is further programmed to amplify the first audio signal and cause transmission of the amplified first audio signal to a hearing interface device configured to provide sound to an ear of the user.

16. The hearing aid system of claim 15 , wherein the hearing interface device includes a speaker associated with an earpiece.

17. The hearing aid system of claim 15 , wherein the hearing interface device includes a bone conduction microphone.

18. The hearing aid system of claim 11 , wherein the wearable camera and the at least one microphone are included in a common housing.

19. The hearing aid system of claim 18 , wherein the at least one processor is included in the common housing.

20. The hearing aid system of claim 18 , wherein the at least one processor is included in a second housing separate from the common housing.

21. The hearing aid system of claim 20 , wherein the at least one processor is configured to receive the plurality of captured images via a wireless link between a transmitter in the common housing and receiver in the second housing.

22. The hearing aid system of claim 20 , wherein the second housing is associated with a paired mobile device.

23. The hearing aid system of claim 11 , wherein the at least one microphone includes a directional microphone.

24. The hearing aid system of claim 11 , wherein the at least one microphone includes a microphone array.

25. The hearing aid system of claim 11 , wherein the at least one processor is further programmed to cause the stored text to be shown on a display.

26. The hearing aid system of claim 25 , wherein the display is associated with a paired mobile device.

27. The hearing aid system of claim 11 , wherein the at least one processor is further programmed to generate a task item based on analysis of the speech associated with the voice of the first individual.

28. The hearing aid system of claim 26 , wherein the at least one processor is further programmed to update a database associated with the user to include the generated task item.

29. The hearing aid system of claim 11 , wherein the at least one processor is further programmed to identify the first individual represented in at least one of the plurality of images.

30. A computer-implemented method for individual identification using a hearing aid system, the method comprising:

receiving audio signals representative of sounds from at least one microphone;

identifying a first audio signal, from among the received audio signals, representative of a voice of a first individual;

transcribing and storing text corresponding to speech associated with the voice of the first individual;

determining whether the first individual is a recognized individual; and

if the first individual is a recognized individual, associating an identifier of the first recognized individual with the stored text corresponding to the speech associated with the voice of the first individual.

31. A non-transitory computer readable storage media, storing program instructions which are executed by at least one processor to perform:

receiving audio signals representative of sounds from at least one microphone;

identifying a first audio signal, from among the received audio signals, representative of a voice of a first individual;

transcribing and storing text corresponding to speech associated with the voice of the first individual;

determining whether the first individual is a recognized individual; and

if the first individual is a recognized individual, associating an identifier of the first recognized individual with the stored text corresponding to the speech associated with the voice of the first individual.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 27, 2023
From: ROSENWEIN, TAL; NATHAN, ROI
To: ORCAM TECHNOLOGIES LTD.
Reel/Frame 062813/0024 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2021
From: WEXLER, YONATAN; SHASHUA, AMNON
To: ORCAM TECHNOLOGIES LTD.
Reel/Frame 055893/0871 →
Continuity (6)
Continuation PCTIB2019001132 · Oct 10, 2019
Provisional Application 62857773 · Jun 5, 2019
Provisional Application 62808317 · Feb 21, 2019
Provisional Application 62746595 · Oct 17, 2018
Provisional Application 62745478 · Oct 15, 2018
Related Publication 20210258703A1 · Aug 19, 2021
Cited By (2)
US 12,450,860 US 12,560,711