IP Library Granted Patent US 11,343,413
Granted Patent B2
US 11,343,413 · App. 17/012,619 · Granted May 24, 2022

Automatically determining a wet microphone condition in a camera

Inventors: Zhinian Jing (Pacifica, CA); Ke Li (San Jose, CA); Erich Tisch (San Francisco, CA); Joyce Rosenbaum (Mountain View, CA); Magnus Hansson (Los Altos, CA)
Assignee: GoPro, Inc.
H04N5/2252G03B17/08H04R1/44H04R3/00H04R3/005H04R5/04H04R29/004H04R2410/05H04R2410/07H04R2499/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,343,413
App. No.
17/012,619
Granted
May 24, 2022
Kind
B2
Abstract

A camera includes a first microphone, a second microphone, and a processor. The processor determines a correlation metric between portions of audio signals obtained from the first and second microphones. The camera includes a memory to store the portions of the audio signals as portions of an output audio signal.

Claims (73)

1. A camera comprising:

a first microphone configured to obtain a first audio signal;

a second microphone configured to obtain a second audio signal;

a processor configured to determine a first correlation metric, wherein the first correlation metric is based on a similarity between a first portion of the first audio signal and a first portion of the second audio signal obtained during a first time interval; and

a memory configured to store the first portion of the first audio signal as a first portion of an output audio signal corresponding to the first time interval on a condition that the first correlation metric exceeds a first predefined threshold.

2. The camera of claim 1 , wherein the processor is further configured to:

determine a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to a determination that the second correlation metric is less than the first predefined threshold, determine whether the first and second microphones are submerged in liquid during the second time interval; and

responsive to a determination that the first and second microphones are submerged in liquid during the second time interval, determine a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

wherein the memory is further configured to store the second portion of the first audio signal or the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

3. The camera of claim 2 , wherein the memory is configured to store the second portion of the first audio signal responsive to a sum of the first noise metric and a bias value being less than the second noise metric.

4. The camera of claim 2 , wherein the memory is configured to store the second portion of the second audio signal responsive to a sum of the first noise metric and a bias value being greater than the second noise metric.

5. The camera of claim 1 , wherein the processor is further configured to:

determine a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to a determination that the second correlation metric is less than the first predefined threshold, determine whether the first microphone is wet and is not submerged in liquid during the second time interval; and

responsive to a determination that the first microphone is wet and is not submerged during the second time interval, store the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval.

6. The camera of claim 1 , wherein the processor is further configured to:

determine a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to a determination that the second correlation metric is less than the first predefined threshold, determine whether the first microphone is wet during the second time interval; and

responsive to a determination that first microphone is wet during the second time interval, determine a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

wherein the memory is further configured to store the second portion of the first audio signal or the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

7. The camera of claim 6 , wherein the memory is configured to store the second portion of the first audio signal responsive to a sum of the first noise metric and a bias value being less than the second noise metric.

8. The camera of claim 6 , wherein the memory is configured to store the second portion of the second audio signal responsive to a sum of the first noise metric and a bias value being greater than the second noise metric.

9. The camera of claim 6 , wherein the processor is configured to determine whether the first microphone is wet, the processor further configured to:

determine a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determine whether a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level exceeding a second predefined threshold or to detecting the wind condition, determine that the first microphone is not wet.

10. The camera of claim 6 , wherein the processor is configured to determine whether the first microphone is wet, the processor further configured to:

determine a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determine whether a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level is less than a second predefined threshold and to not detecting the wind condition, determine that the first microphone is wet.

11. A method for generating an output audio signal in a camera, the method comprising:

obtaining a first audio signal from a first microphone;

obtaining a second audio signal from a second microphone;

determining a first correlation metric, wherein the first correlation metric is based on a similarity between a first portion of the first audio signal and a first portion of the second audio signal captured during a first time interval;

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval; and

responsive to the first correlation metric exceeding a first predefined threshold, storing the first portion of the first audio signal as a first portion of the output audio signal corresponding to the first time interval.

12. The method of claim 11 , further comprising:

responsive to determining that the second correlation metric is less than the first predefined threshold, determining whether the first and second microphones are submerged in liquid during the second time interval;

responsive to determining that the first and second microphones are submerged in liquid during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

storing the second portion of the first audio signal or the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

13. The method of claim 11 , further comprising:

responsive to determining that the second correlation metric is less than the first predefined threshold, determining whether the first microphone is wet and is not submerged in liquid during the second time interval; and

responsive to determining that the first microphone is wet and is not submerged during the second time interval, storing the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval.

14. The method of claim 11 , further comprising:

responsive to determining that the second correlation metric is less than the first predefined threshold, determining whether the first microphone is wet during the second time interval;

responsive to determining that the first microphone is wet during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

storing the second portion of the first audio signal or the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

15. The method of claim 14 , wherein determining whether the first microphone is wet comprises:

determining a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determining whether a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level is less than a second predefined threshold and to not detecting the wind condition, determining that the first microphone is wet.

16. A non-transitory computer-readable medium storing instructions, the instructions when executed by a processor causing the processor to perform steps including:

determining a first correlation metric, wherein the first correlation metric is based on a similarity between a first audio signal and a second audio signal captured during a first time interval; and

storing a first portion of the first audio signal as a first portion of an output audio signal corresponding to the first time interval.

17. The non-transitory computer-readable medium of claim 16 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to the second correlation metric being less than the first predefined threshold, determining whether the first and second microphones are submerged in liquid during the second time interval;

responsive to determining that the first and second microphones are submerged in liquid during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

storing the second portion of the first audio signal or the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

18. The non-transitory computer-readable medium of claim 16 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric is less than the first predefined threshold, determining whether the first microphone is wet and is not submerged in liquid during the second time interval; and

responsive to determining that the first microphone is wet and is not submerged in liquid during the second time interval, storing the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval.

19. The non-transitory computer-readable medium of claim 16 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric is less than the first predefined threshold, determining whether the first microphone is wet during the second time interval;

responsive to determining that first microphone is wet during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

storing the second portion of the first audio signal or the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

20. The non-transitory computer-readable medium of claim 19 , wherein determining whether the first microphone is wet comprises:

determining a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determining whether a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level being less than a second predefined threshold and to not detecting the wind condition, determining that the first microphone is wet.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2021
From: JING, ZHINIAN; TISCH, ERICH; ROSENBAUM, JOYCE; HANSSON, MAGNUS
To: GOPRO, INC.
Reel/Frame 056090/0167 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2021
From: DSP CONCEPTS, LLC
To: GOPRO, INC.
Reel/Frame 056090/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2021
From: LI, KE
To: DSP CONCEPTS, LLC
Reel/Frame 056102/0164 →
Continuity (5)
Continuation 16440430 · Jun 13, 2019
Continuation 15674355 · Aug 10, 2017
Continuation 15083266 · Mar 28, 2016
Provisional Application 62188450 · Jul 2, 2015
Related Publication 20200404126A1 · Dec 24, 2020