IP Library › Granted Patent US 11,809,774
Granted Patent B1
US 11,809,774 · App. 16/890,953 · Granted Nov 7, 2023

Privacy with extra-aural speakers

Inventors: Nikolas T. Vitt (Redwood City, CA); Jonathan D. Sheaffer (San Jose, CA); Neal D. Evans (Sunnyvale, CA); Christopher T. Eubank (Mountain View, CA); Jae Hwang Lee (Los Gatos, CA)
Assignee: Apple Inc.
G06F3/165G06F3/017H04R3/00H04R2430/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,809,774
App. No.
16/890,953
Filed
Jun 2, 2020
Granted
Nov 7, 2023
Kind
B1
Art Unit
2692
USPC
700/94
Abstract

A method performed by a processor of a computer system including a headset that is to be worn on a head of a user. The method drives a speaker of the headset with an input audio signal to output sound into an environment. The method determines that the speaker is at least partially covered with a cupped hand. In response to determining that the speaker is at least partially covered with the cupped hand, applying a gain to the input audio signal to reduce an output sound level of the speaker.

Claims (72)

1. A method performed by a processor of a computer system comprising a headset, the method comprising:

driving an extra-aural speaker of the headset with an input audio signal to output sound into an environment;

determining that the extra-aural speaker is at least partially covered;

in response to determining that the extra-aural speaker is at least partially covered,

determining a gain based on an estimated change to a perceived sound level of the sound caused by the at least partial covering of the extra-aural speaker; and

producing an adjusted audio signal by applying the gain to the input audio signal, which when used to drive the extra-aural speaker produces sound that compensates for the estimated change to the perceived sound level.

2. The method of claim 1 , wherein determining that the extra-aural speaker is at least partially covered comprises

obtaining a microphone signal produced by a microphone in the headset; and

processing the microphone signal to estimate a change to a sound characteristic of the microphone signal that is indicative of the extra-aural speaker being at least partially covered.

3. The method of claim 2 , wherein the sound characteristic is the perceived sound level, wherein the change is an increase of the perceived sound level above a threshold level.

4. The method of claim 2 , wherein determining that the extra-aural speaker is at least partially covered comprises determining that the extra-aural speaker is at least partially covered by a cupped hand according to the change to the sound characteristic.

5. The method of claim 1 further comprising estimating a sound field that is produced by the extra-aural speaker while at least partially covered, wherein the gain is determined based on whether a user is within the estimated sound field.

6. The method of claim 5 further comprising determining whether the user is within the estimated sound field by

obtaining image data from a camera that represents a scene of the environment;

performing an object recognition algorithm upon the image data to identify the user within the scene of the environment;

determining a location of the user with respect to the headset;

determining a privacy threshold based on the location of the user with respect to the headset; and

determining whether the estimated sound field is outside or inside the privacy threshold.

7. The method of claim 6 , wherein the gain is determined in response to determining that the sound field is outside the privacy threshold.

8. The method of claim 6 further comprising obtaining a microphone signal produced by a microphone of the headset,

wherein in response to determining that the estimated sound field is inside the privacy threshold, the method comprises applying an equalization filter upon the input audio signal to compensate for a change to spectral content of the microphone signal due to the extra-aural speaker being at least partially covered.

9. The method of claim 1 further comprising:

producing a spatially rendered audio signal by applying a spatial audio filter upon the input audio signal; and

driving the extra-aural speaker using the spatially rendered audio signal to produce binaural sound,

wherein in response to determining that the extra-aural speaker is at least partially covered, the method comprises ceasing to apply the spatial audio filter upon the input audio signal.

10. The method of claim 1 , wherein the input audio signal is a first input audio signal and the extra-aural speaker is a first extra-aural speaker, wherein the method further comprises driving a second extra-aural speaker of the headset with a second input audio signal to output sound into the environment,

wherein in response to determining that the first extra-aural speaker is at least partially covered, the method further comprises ceasing to drive the second extra-aural speaker with the second input audio signal, while the first extra-aural speaker is at least partially covered.

11. The method of claim 1 , wherein the applied gain compensates for the estimated change to the input sound level by maintaining the input sound level from before the extra-aural speaker was at least partially covered.

12. A method performed by a processor of a computer system comprising a headset that is to be worn on a head of a user, the method comprising:

obtaining a left audio signal and a right audio signal;

driving a left extra-aural speaker with the left audio signal and a right extra-aural speaker with the right audio signal;

determining that the left extra-aural speaker of the headset is at least partially covered;

in response to determining that the left extra-aural speaker is at least partially covered, processing the left audio signal to produce a left adjusted audio signal based on an estimated amount of occlusion caused by the at least partial covering of the left extra-aural speaker of the headset; and

driving the left extra-aural speaker with the left adjusted audio signal to output sound that compensates for the estimated amount of occlusion.

13. The method of claim 12 further comprises, before determining that the left extra-aural speaker is at least partially covered, driving the left extra-aural speaker with the left audio signal to output sound, wherein determining that the left extra-aural speaker is at least partially covered comprises

obtaining a microphone signal captured by a microphone of the headset that contains sound of the left audio signal; and

determining, using the microphone signal, a frequency response between the microphone and the left extra-aural speaker that is indicative of the left extra-aural speaker being at least partially covered.

14. The method of claim 13 further comprising determining the amount of occlusion caused by at least partial covering of the left extra-aural speaker based on the frequency response, wherein processing the left audio signal comprises

producing a gain to compensate for the amount of occlusion; and

applying the gain to the left audio signal to produce the left adjusted audio signal.

15. The method of claim 14 , wherein the gain compensates for the amount of occlusion by reducing an output sound level of the left extra-aural speaker in order to maintain an input sound level of the microphone signal from before determining that the left extra-aural speaker is at least partially covered.

16. The method of claim 14 , wherein processing the left audio signal further comprises

determining an equalization filter to compensate for the amount of occlusion; and

spectrally shaping the left audio signal to produce the left adjusted audio signal according to the equalization filter in order to maintain spectral content of the microphone signal from before determining that the left extra-aural speaker was at least partially covered.

17. The method of claim 12 further comprising

determining the amount of occlusion caused by the at least partial covering of the left extra-aural speaker;

determining a sound field to be produced by the left extra-aural speaker and the right extra-aural speaker while being driven by the left audio signal and the right audio signal according to the amount of occlusion;

determining a privacy threshold that represents a distance from the headset that sound outputted by the left and right extra-aural speakers is only perceived by the user; and

determining whether the privacy threshold exceeds the sound field,

wherein in response to determining that the privacy threshold exceeds the sound field processing the left audio signal comprises compensating for the amount of occlusion by producing the left adjusted audio signal which when used to drive the left extra-aural speaker while covered has a perceived sound level that is similar to a perceived sound level of the left audio signal when used to drive the left extra-aural speaker while not covered.

18. The method of claim 12 further comprising

processing, in response to determining that the left extra-aural speaker is at least partially covered, the right audio signal to produce a right adjusted audio signal, wherein the left audio signal is processed differently than the right audios signal; and

driving the right extra-aural speaker with the right adjusted audio signal.

19. The method of claim 18 further comprises determining that the right speaker of the headset is at least partially covered, wherein processing the left audio signal and the right audio signal comprises

producing a left gain signal according to the left extra-aural speaker being at least partially covered and producing a right, different, gain signal according to the right extra-aural speaker being at least partially covered; and

applying the left gain signal to the left audio signal to produce the left adjusted audio signal, which when used to drive the left extra-aural speaker causes the left extra-aural speaker to output the sound of the left adjusted audio signal at a first sound level and applying the right gain signal to the right audio signal to produce the right adjusted audio signal, which when used to drive the right extra-aural speaker causes the right extra-aural speaker to output the sound of the right adjusted audio signal at a second, different, sound level,

wherein, while driving the left extra-aural speaker with the left adjusted audio signal and driving the right extra-aural speaker with the right adjusted audio signal, an input sound level of the sound of both signals at each of left and right ears of the user is the same.

20. A headset comprising:

an extra-aural speaker;

a processor; and

memory having stored therein instructions that when executed by the processor causes the headset to

drive the extra-aural speaker with an audio signal to output sound into an environment;

determine that the extra-aural speaker is at least partially covered;

determine an estimated change to a perceived sound level of the sound caused by the at least partial covering of the extra-aural speaker; and

responsive to determining that the extra-aural speaker is at least partially covered,

produce an adjusted audio signal, which when used to drive the extra-aural speaker produces sound that compensates for a change to a perceived sound level of the sound of the audio signal caused by the at least partial covering of the extra-aural speaker.

21. The headset of claim 20 , wherein the instructions to determine that the extra-aural speaker is at least partially covered comprises instructions to

obtain a microphone signal produced by a microphone in the headset; and

process the microphone signal to estimate a change to a sound characteristic of the microphone signal that is indicative of the extra-aural speaker being at least partially covered.

22. The headset of claim 21 , wherein the instructions to determine that the extra-aural speaker is at least partially covered comprises instructions to determine that the extra-aural speaker is at least partially covered by a cupped hand according to the change to the sound characteristic.

23. The headset of claim 20 , wherein the audio signal is a first audio signal and the extra-aural speaker is a first extra-aural speaker, wherein the memory has further instructions to drive a second extra-aural speaker of the headset with a second audio signal to output sound into the environment,

wherein the memory has further instructions to, in response to determining that the first extra-aural speaker is at least partially covered, cease to drive the second extra-aural speaker with the second audio signal, while the first extra-aural speaker is at least partially covered.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2020
From: VITT, NIKOLAS T.; SHEAFFER, JONATHAN D.; EVANS, NEAL D.; EUBANK, CHRISTOPHER T.; LEE, JAE HWANG
To: APPLE INC.
Reel/Frame 052824/0747 →
Continuity (1)
Provisional Application 62857707 · Jun 5, 2019
Cited By (4)
US 12,205,578 US 12,330,078 US 12,647,736 US 12,732,769