IP Library › Granted Patent US 11,990,129
Granted Patent B2
US 11,990,129 · App. 16/915,951 · Granted May 21, 2024

Display eyewear with auditory enhancement

Inventors: Jerome A. Legerton (Jupiter, FL); Jay Marsh (Bonsall, CA); Stephen Willey (Blaine, WA)
Assignee: Innovega, Inc.
G10L15/26G06F3/013G06F3/167G10L15/25G10L21/10G10L2021/065
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,990,129
App. No.
16/915,951
Granted
May 21, 2024
Kind
B2
Abstract

Some embodiments provide display eyewear with auditory enhancement. In general, one aspect disclosed features a head-wearable apparatus comprising: a microphone; a display panel visible to the wearer; a gaze tracker configured to determine a direction of a gaze of a wearer of the head-wearable apparatus; and a controller configured to: extract speech from sound collected by the microphone from the determined direction, and present the extracted speech on the display panel.

Claims (61)

1. A head-wearable apparatus comprising:

a first microphone, located on a front end of the apparatus facing a front side of the wearer, which collects first sounds emanating from the front of a head of a wearer of the head-wearable apparatus;

a display panel visible to the wearer;

a gaze tracker configured to determine a direction of a gaze of the wearer of the head-wearable apparatus;

multiple second microphones, located on both sides and toward a back end of the apparatus and of the wearer, which collect second sounds emanating from the sides and rear of the head of the wearer;

an auditory transducer; and

a controller configured to:

extract speech from the first sounds collected by the first microphone from the determined direction,

present the extracted speech on the display panel,

provide, to the auditory transducer, second audio representing the second sounds collected by the multiple second microphones responsive to the second sounds representing a predetermined keyword, wherein the auditory transducer renders the second audio, and

present the keyword on the display panel in a location that indicates a direction from which the second sounds emanate.

2. The head-wearable apparatus of claim 1 , wherein the controller is further configured to present the extracted speech as text on the display panel.

3. The head-wearable apparatus of claim 1 , wherein the controller is further configured to present the extracted speech as hand signs on the display panel.

4. The head-wearable apparatus of claim 1 :

wherein the controller is further configured to provide, to the auditory transducer, audio representing the first sounds collected by the first microphone from the determined direction, wherein the auditory transducer renders the audio representing the first sounds collected by the first microphone from the determined direction.

5. The head-wearable apparatus of claim 4 , further comprising:

a hearing aid system comprising the auditory transducer.

6. The head-wearable apparatus of claim 1 : wherein the controller is further configured to provide, to the auditory transducer, audio representing the extracted first and/or second speech, wherein the auditory transducer renders the audio.

7. The head-wearable apparatus of claim 1 , wherein the head-wearable apparatus is a pair of eyeglasses.

8. A head-wearable apparatus comprising:

a first microphone, located on a front end of the apparatus facing a front side of the wearer, which collects first sounds emanating from the front of a head of a wearer of the head-wearable apparatus;

a display panel visible to the wearer;

a gaze tracker configured to determine a direction of a gaze of the wearer of the head-wearable apparatus;

multiple second microphones, located on both sides and toward a back end of the apparatus and of the wearer, which collect second sounds emanating from the sides and rear of the head of the wearer; and

a controller configured to:

extract first speech from the first sounds collected by the first microphone from the determined direction,

present the extracted first speech on the display panel, and

responsive to the second sounds representing a predetermined keyword,

extracting second speech from the second sounds collected by the multiple second microphones, and

presenting the extracted second speech on the display panel in a location that indicates a direction from which the second sounds emanate.

9. The head-wearable apparatus of claim 8 , wherein the controller is further configured to present the extracted first and/or second speech as text on the display panel.

10. The head-wearable apparatus of claim 8 , wherein the controller is further configured to present the extracted first and/or second speech as hand signs on the display panel.

11. The head-wearable apparatus of claim 8 , further comprising:

an auditory transducer;

wherein the controller is further configured to provide, to the auditory transducer, audio representing the first sound collected by the first microphone from the determined direction, wherein the auditory transducer renders the audio representing the first sound collected by the first microphone from the determined direction.

12. The head-wearable apparatus of claim 11 , further comprising:

a hearing aid system comprising the auditory transducer.

13. The head-wearable apparatus of claim 8 , further comprising:

an auditory transducer;

wherein the controller is further configured to provide, to the auditory transducer, audio representing the extracted first and/or second speech, wherein the auditory transducer renders the audio.

14. The head-wearable apparatus of claim 8 , wherein the head-wearable apparatus is a pair of eyeglasses.

15. A non-transitory machine-readable storage medium encoded with instructions executable by a hardware processor of a computing component, the machine-readable storage medium comprising instructions to cause the hardware processor to perform a method for a head-wearable apparatus, the method comprising:

determining a direction of a gaze of a wearer of the head-wearable apparatus;

collecting first sounds emanating from the determined direction using a first microphone, located on a front end of the apparatus facing a front side of the wearer, which collects the first sounds emanating from the front of a head of a wearer of the head-wearable apparatus;

extracting first speech from the collected first sounds;

presenting the extracted first speech on a display panel of the head-wearable apparatus, wherein the display panel is visible to the wearer;

collecting second sounds emanating from the sides and rear of the head of the wearer using a second microphone, located on a side and toward a back end of the apparatus and of the wearer, which collects the second sounds emanating from the sides and rear of the wearer's head;

providing audio representing the second sounds to an auditory transducer of the head-wearable apparatus responsive to the sounds representing a predetermined keyword, wherein the auditory transducer renders the second audio, and

presenting the keyword on the display panel in a location that indicates a direction from which the second sounds emanate.

16. The non-transitory machine-readable storage medium of claim 15 , wherein the head-wearable apparatus is a pair of eyeglasses.

17. The non-transitory machine-readable storage medium of claim 15 , the method further comprising:

presenting the extracted first and/or second speech as text on the display panel.

18. The non-transitory machine-readable storage medium of claim 15 , the method further comprising:

projecting the extracted first speech onto the display panel using an off-axis projector;

wherein the display panel is a transflective diffuser.

19. The non-transitory machine-readable storage medium of claim 15 , the method further comprising:

presenting the extracted first and/or second speech as hand signs on the display panel.

20. The non-transitory machine-readable storage medium of claim 15 , the method further comprising:

providing audio representing the collected first sounds to an auditory transducer of the head-wearable apparatus, wherein the auditory transducer renders the audio.

21. The non-transitory machine-readable storage medium of claim 15 , the method further comprising:

providing audio representing the extracted first and/or second speech to an auditory transducer of the head-wearable apparatus, wherein the auditory transducer renders the audio.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2023
From: LEGERTON, JEROME A.; MARSH, JAY; WILLEY, STEPHEN
To: INNOVEGA, INC.
Reel/Frame 063185/0313 →
Continuity (1)
Related Publication 20210407513A1 · Dec 30, 2021
Cited By (1)
US 12,578,758