IP Library Granted Patent US 10,944,936
Granted Patent B2
US 10,944,936 · App. 16/550,566 · Granted Mar 9, 2021

Beam forming for microphones on separate faces of a camera

Inventors: Zhinian Jing (Pacifica, CA); Joyce Rosenbaum (Mountain View, CA)
Assignee: GoPro, Inc.
H04N5/607H04N5/225H04R5/027H04S7/305H04N5/232H04N5/23264H04R2203/12H04R2499/11H04S2400/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,944,936
App. No.
16/550,566
Granted
Mar 9, 2021
Kind
B2
Abstract

A camera system capable of capturing images of an event in a dynamic environment includes two microphones configured to capture stereo audio of the event. The microphones are on orthogonal surfaces of the camera system. Because the microphones are on orthogonal surfaces of the camera system, the camera body can impact the spatial response of the two recorded audio channels differently, leading to degraded stereo recreation if standard beam forming techniques are used. The camera system includes tuned beam forming techniques to generate multi-channel audio that more accurately recreates the stereo audio by compensating for the shape of the camera system and the orientation of microphones on the camera system. The tuned beam forming techniques include optimizing a set of beam forming parameters, as a function of frequency, based on the true spatial response of the recorded audio signals.

Claims (36)

1. A camera system, comprising:

a first microphone on a first surface;

a second microphone on a second surface, wherein the second surface is substantially orthogonal to the first surface; and

a processor, the processor coupled with a memory storing instructions that when executed causes the processor to:

apply a set of beam forming parameters in audio channels associated with the first and second microphones;

process the audio channels using the set of beam forming parameters to generate virtual microphone channels; and

combine the virtual microphone channels into an audio stream.

2. The camera system of claim 1 , wherein the set of beam forming parameters applied to the audio channels is associated with a linear distance between the first microphone and the second microphone across a surface of a camera body.

3. The camera system of claim 1 , wherein the set of beam forming parameters applied to the audio channels is associated with a linear distance between the first microphone and the second microphone through a camera body.

4. The camera system of claim 1 , wherein the set of beam forming parameters applied to the audio channels is based on maximizing a noise ratio between opposing hemispheres of a spatial response of the audio channels.

5. The camera system of claim 1 , wherein the set of beam forming parameters are independently applied to separate frequency sub-bands of the audio channels based on a shape of a camera body and a frequency range of the separate frequency sub-bands.

6. The camera system of claim 1 , wherein the set of beam forming parameters are tuned for temporal, spectral, and spatial characteristics and access gain and delay values stored in the memory associated with characteristics and a shape of a camera body.

7. An electronic device, comprising:

a device body;

a first microphone on a first surface of the electronic device;

a second microphone on a second surface of the electronic device, wherein the second surface is substantially orthogonal to the first surface;

an audio assembly; and

a processor, the processor coupled with a memory storing instructions that when executed causes the processor to:

apply a set of tuned parameters in audio channels associated with orthogonality of the first surface and the second surface;

process the audio channels using the set of tuned parameters to generate virtual channels; and

combine the virtual channels into an audio stream.

8. The electronic device of claim 7 , wherein the set of tuned parameters applied to the audio channels is associated with a linear distance between the first microphone and the second microphone across a surface of the electronic device.

9. The electronic device of claim 7 , wherein the set of tuned parameters applied to the audio channels is associated with a linear distance between the first microphone and the second microphone through the device body.

10. The electronic device of claim 7 , wherein the set of tuned parameters applied to the audio channels is based on maximizing a noise ratio between opposing hemispheres of a spatial response of the audio channels.

11. The electronic device of claim 7 , wherein the set of tuned parameters are independently applied to separate frequency sub-bands of the audio channels based on a shape of the device body and a frequency range of the separate frequency sub-bands.

12. The electronic device of claim 7 , wherein the set of tuned parameters are beam forming parameters that are tuned for temporal, spectral, and spatial characteristics and access gain and delay values stored in the memory associated with the device body.

13. A method, comprising:

capturing audio channels from a first microphone of a camera and a second microphone of the camera, wherein the second microphone is substantially orthogonal to the first microphone;

applying a set of tuned beam forming parameters to a set of frequency sub-bands of the audio channels associated with the first microphone and the second microphone;

processing the audio channels using the set of tuned beam forming parameters to generate virtual microphone channels; and

combining the virtual microphone channels into an audio stream that is configured for stereo audio playback.

14. The method of claim 13 , wherein the set of tuned beam forming parameters applied to the audio channels is associated with a linear distance between the first microphone and the second microphone across a surface of a camera body.

15. The method of claim 13 , wherein the set of tuned beam forming parameters applied to the audio channels is associated with a linear distance between the first microphone and the second microphone through a camera body.

16. The method of claim 13 , wherein the set of tuned beam forming parameters applied to the audio channels is based on maximizing a noise ratio between opposing hemispheres of a spatial response of the audio channels.

17. The method of claim 13 , wherein the set of tuned beam forming parameters are independently applied to separate frequency sub-bands of the audio channels based on a shape of a camera body and a frequency range of the separate frequency sub-bands.

18. The method of claim 13 , wherein the set of tuned beam forming parameters are tuned for temporal, spectral, and spatial characteristics and access gain and delay values stored in memory associated with characteristics of a camera body.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2020
From: JING, ZHINIAN; ROSENBAUM, JOYCE
To: GOPRO, INC.
Reel/Frame 051627/0731 →
Continuity (4)
Continuation 16154079 · Oct 8, 2018
Continuation 15701426 · Sep 11, 2017
Provisional Application 62395953 · Sep 16, 2016
Related Publication 20200021771A1 · Jan 16, 2020