IP Library Granted Patent US 10,771,660
Granted Patent B2
US 10,771,660 · App. 16/440,430 · Granted Sep 8, 2020

Automatically determining a wet microphone condition in a camera

Inventors: Zhinian Jing (Pacifica, CA); Ke Li (San Jose, CA); Erich Tisch (San Francisco, CA); Joyce Rosenbaum (Mountain View, CA); Magnus Hansson (Los Altos, CA)
Assignee: GoPro, Inc.
H04N5/2252G03B17/08H04R1/44H04R3/00H04R3/005H04R5/04H04R29/004H04R2410/05H04R2410/07H04R2499/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,771,660
App. No.
16/440,430
Granted
Sep 8, 2020
Kind
B2
Abstract

An audio capture system for a sports camera includes at least one “enhanced” microphone and at least one “reference” microphone. The enhanced microphone includes a drainage enhancement feature to enable water to drain from the microphone more quickly than the reference microphone. A microphone selection controller selects between the microphones based on a microphone selection algorithm to enable high quality in conditions where the sports camera transitions in and out of water during activities such as surfing, water skiing, swimming, or other wet environments.

Claims (84)

1. An audio capture system comprising:

a first microphone;

a second microphone;

a processor; and

a memory storing instructions that when executed by the processor cause the processor to perform steps including:

a non-transitory computer-readable medium storing instructions for generating an output audio signal, the instructions when executed by the processor causing the processor to perform steps including:

determining a first correlation metric between a first portion of a first audio signal of the first microphone and a first portion of a second audio signal of the second microphone captured during a first time interval, the first correlation metric representing a similarity between the first portion of the first audio signal and the first portion of the second audio signal during the first time interval; and

responsive to the first correlation metric exceeding a first predefined threshold, storing the first portion of the first audio signal as a first portion of the output audio signal corresponding to the first time interval.

2. The audio capture system of claim 1 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining if the first and second microphones are submerged in liquid during the second time interval;

responsive to determining that the first and second microphones are submerged in liquid during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

selecting between storing the second portion of the first audio signal and the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

3. The audio capture system of claim 2 , wherein selecting between storing the second portion of the first audio signal and the second portion of the second audio signal comprises:

selecting the second portion of the first audio signal responsive to a sum of the first noise metric and a bias value being less than the second noise metric.

4. The audio capture system of claim 2 , wherein selecting between storing the second portion of the first audio signal and the second portion of the second audio signal comprises:

selecting the second portion of the second audio signal responsive to a sum of the first noise metric and a bias value being greater than the second noise metric.

5. The audio capture system of claim 1 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining whether the first microphone is wet but is not submerged in liquid during the second time interval; and

responsive to determining that the first microphone is wet but is not submerged during the second time interval, storing the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval.

6. The audio capture system of claim 1 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining whether the first microphone is wet during the second time interval;

responsive to determining that first microphone is wet during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

selecting between storing the second portion of the first audio signal and the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

7. The audio capture system of claim 6 , wherein selecting between storing the second portion of the first audio signal and the second portion of the second audio signal comprises:

selecting the second portion of the first audio signal responsive to a sum of the first noise metric and a bias value being less than the second noise metric.

8. The audio capture system of claim 6 , wherein selecting between storing the second portion of the first audio signal and the second portion of the second audio signal comprises:

selecting the second portion of the second audio signal responsive to a sum of the first noise metric and a bias value being greater than the second noise metric.

9. The audio capture system of claim 6 , wherein determining if the first microphone is wet comprises:

determining a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determining if a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level exceeding a second predefined threshold or to detecting the wind condition, determining that the first microphone is not wet.

10. The audio capture system of claim 6 , wherein determining if the first microphone is wet comprises:

determining a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determining if a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level not exceeding a second predefined threshold and to not detecting the wind condition, determining that the first microphone is wet.

11. A method for generating an output audio signal in an audio capture system, the method comprising:

receiving a first audio signal from a first microphone of the audio capture system;

receiving a second audio signal from a second microphone of the audio capture system;

determining a first correlation metric between a first portion of the first audio signal and a first portion of the second audio signal captured during a first time interval, the first correlation metric representing a similarity between the first portion of the first audio signal and the first portion of the second audio signal during the first time interval; and

responsive to the first correlation metric exceeding a first predefined threshold, storing the first portion of the first audio signal as a first portion of the output audio signal corresponding to the first time interval.

12. The method of claim 11 , further comprising:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to the second correlation metric not exceeding the first predefined threshold, determining if the first and second microphones are submerged in liquid during the second time interval;

responsive to determining that the first and second microphones are submerged in liquid during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

selecting between storing the second portion of the first audio signal and the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

13. The method of claim 11 , further comprising:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining whether the first microphone is wet but is not submerged in liquid during the second time interval; and

responsive to determining that the first microphone is wet but is not submerged during the second time interval, storing the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval.

14. The method of claim 11 , further comprising:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining whether the first microphone is wet during the second time interval;

responsive to determining that the first microphone is wet during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

selecting between storing the second portion of the first audio signal and the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

15. The method of claim 14 , wherein determining if the first microphone is wet comprises:

determining a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determining if a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level not exceeding a second predefined threshold and to not detecting the wind condition, determining that the first microphone is wet.

16. A non-transitory computer-readable medium storing instructions, the instructions when executed by a processor causing the processor to perform steps including:

receiving a first audio signal from a first microphone that includes a drain;

receiving a second audio signal from a second microphone;

determining a first correlation metric between a first portion of the first audio signal and a first portion of the second audio signal captured during a first time interval; and

responsive to the first correlation metric exceeding a first predefined threshold, storing the first portion of the first audio signal as a first portion of an output audio signal corresponding to the first time interval.

17. The non-transitory computer-readable medium of claim 16 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to the second correlation metric not exceeding the first predefined threshold, determining if the first and second microphones are submerged in liquid during the second time interval;

responsive to determining that the first and second microphones are submerged in liquid during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

selecting between storing the second portion of the first audio signal and the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

18. The non-transitory computer-readable medium of claim 16 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining whether the first microphone is wet but is not submerged in liquid during the second time interval; and

responsive to determining that the first microphone is wet but is not submerged in liquid during the second time interval, storing the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval.

19. The non-transitory computer-readable medium of claim 16 , the instructions when executed further causing the processor to perform steps including:

determining a second correlation metric between a second portion of the first audio signal and a second portion of the second audio signal captured during a second time interval;

responsive to determining that the second correlation metric does not exceed the first predefined threshold, determining whether the first microphone is wet during the second time interval;

responsive to determining that first microphone is wet during the second time interval, determining a first noise metric for the second portion of the first audio signal and a second noise metric for the second portion of the second audio signal during the second time interval; and

selecting between storing the second portion of the first audio signal and the second portion of the second audio signal as a second portion of the output audio signal corresponding to the second time interval based on the first and second noise metrics.

20. The non-transitory computer-readable medium of claim 19 , wherein determining if the first microphone is wet comprises:

determining a first average signal level of the second portion of the first audio signal and a second average signal level for the second portion of the second audio signal;

determining if a wind condition is detected during the second time interval; and

responsive to a ratio of the first average signal level to the second average signal level not exceeding a second predefined threshold and to not detecting the wind condition, determining that the first microphone is wet.

Assignments (7)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2019
From: DSP CONCEPTS, LLC
To: GOPRO, INC.
Reel/Frame 049536/0545 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2019
From: LI, KE
To: DSP CONCEPTS, LLC
Reel/Frame 049537/0054 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2019
From: JING, ZHINIAN; TISCH, ERICH; ROSENBAUM, JOYCE; HANSSON, MAGNUS
To: GOPRO, INC.
Reel/Frame 049536/0621 →
Continuity (4)
Continuation 15674355 · Aug 10, 2017
Continuation 15083266 · Mar 28, 2016
Provisional Application 62188450 · Jul 2, 2015
Related Publication 20190364177A1 · Nov 28, 2019