IP Library › Granted Patent US 12,490,040
Granted Patent B2
US 12,490,040 · App. 18/204,630 · Granted Dec 2, 2025

Direction of audio arrival from Ambisonics channels using azimuth and elevation segments

Inventors: Hakim Mesiwala (Cupertino, CA); Eric Steven Penrod (Brentwood, CA); Erich Tisch (San Francisco, CA); Mark Hardin (Santa Rosa, CA); Timothy Dick (San Francisco, CA); William Edward MacDonald (Oceanside, CA)
Assignee: GoPro, Inc.
H04S7/30H04R1/028H04R3/005H04R5/027H04S3/008H04R2499/11H04S2400/01H04S2400/15H04S2420/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,490,040
App. No.
18/204,630
Granted
Dec 2, 2025
Kind
B2
Abstract

Devices and methods for determining a direction of audio arrival from Ambisonics channels using azimuth and elevation segments is described herein. A method includes generating multiple blocks of samples from Ambisonics signals for a time interval, determining an azimuth angle estimate and an elevation angle estimate for the time interval when a defined number of blocks in the multiple blocks of samples are valid, generating the azimuth angle estimate based on maximum number of azimuth angle estimates present in an azimuth segment amongst a defined number of azimuth segments, and generating the elevation angle estimates based on maximum number of elevation angle estimates present in an elevation segment amongst a defined number of elevation segments, where the direction of arrival of the Ambisonics signals is based on the azimuth angle estimate and the elevation angle estimate.

Claims (64)

1 . A method comprising:

generating multiple blocks of samples from Ambisonics signals for a time interval;

determining an azimuth angle estimate and an elevation angle estimate for the time interval when a defined number of blocks in the multiple blocks of samples are valid;

generating the azimuth angle estimate based on maximum number of azimuth angle estimates present in an azimuth segment amongst a defined number of azimuth segments;

generating the elevation angle estimate based on maximum number of elevation angle estimates present in an elevation segment amongst a defined number of elevation segments;

determining, for the azimuth angle estimate, an azimuth angle estimation confidence level based on a ratio of the maximum number of azimuth angle estimates present in the azimuth segment and a total number of azimuth angle estimates present in the defined number of azimuth segments; and

determining, for the elevation angle estimation, an elevation angle estimation confidence level based on a ratio of the maximum number of elevation angle estimates present in the elevation segment and a total number of elevation angle estimates present in the defined number of elevation segments,

wherein a direction of arrival of the Ambisonics signals is based on the azimuth angle estimate and the elevation angle estimate.

2 . The method of claim 1 , further comprising:

transforming, for a block of samples, at least the block of samples into a defined number of frequency domain bins; and

performing, in a subset of the defined number of frequency domain bins, a spectral analysis to determine whether the block of samples is valid.

3 . The method of claim 2 , wherein a frequency domain bin has a spectral value which is representative of an energy level in the frequency domain bin.

4 . The method of claim 3 , wherein the performing is further comprising:

designating the block of samples as valid if, for a W channel in the Ambisonics signals, a total energy across the subset of the defined number of frequency domain bins exceeds a defined active signal threshold.

5 . The method of claim 4 , wherein the performing is further comprising:

determining, for an X channel in the Ambisonics signals, an X signal threshold based on an X channel energy contribution to a W channel energy;

determining, for a Y channel in the Ambisonics signals, a Y signal threshold based on an Y channel energy contribution to the W channel energy;

determining, for a Z channel in the Ambisonics signals, a Z signal threshold based on an Z channel energy contribution to the W channel energy; and

designating a frequency domain bin in the subset of the defined number of frequency domain bins as a key frequency domain bin if at least one of an X channel energy value exceeds the X signal threshold, a Y channel energy value exceeds the Y signal threshold, or a Z channel energy value exceeds the Z signal threshold.

6 . The method of claim 1 , further comprising:

transforming a block of samples into a defined number of frequency domain bins; and

designating the block of samples as valid if, for a W channel in the Ambisonics signals, a total energy across a subset of the defined number of frequency domain bins exceeds a defined active signal threshold.

7 . The method of claim 1 , wherein the defined number of azimuth segments defines a 360° azimuth space.

8 . The method of claim 7 , wherein the defined number of elevation segments defines an elevation space between +90° to −90°.

9 . The method of claim 1 , further comprising:

applying rotation parameters to the Ambisonics signals prior to the determining the azimuth angle estimate and the elevation angle estimate when the method is performed separately or remote from an image capture device.

10 . An image capture device, comprising:

a plurality of microphones; and

a processor configured to receive Ambisonics signals for a time interval from the plurality of microphones, wherein the processor is further configured to:

divide the time interval into a plurality of blocks;

determine an azimuth angle estimate and an elevation angle estimate for the time interval when a subset of blocks in the plurality of blocks are valid;

identify the azimuth angle estimate based on a maximum number of azimuth angle estimates present in an azimuth segment from a plurality of azimuth segments;

identify the elevation angle estimate based on a maximum number of elevation angle estimates present in an elevation segment from a plurality of elevation segments;

report, for the azimuth angle estimate, an azimuth angle estimate confidence level based on a ratio of the maximum number of azimuth angle estimates present in the azimuth segment and a total number of azimuth angle estimates present in the plurality of azimuth segments; and

report, for the elevation angle estimate, an elevation angle estimate confidence level based on a ratio of the maximum number of elevation angle estimates present in the elevation segment and a total number of elevation angle estimates present in the plurality of elevation segments,

wherein a direction of arrival of the Ambisonics signals is based on the azimuth angle estimate and the elevation angle estimate.

11 . The image capture device of claim 10 , wherein the processor is further configured to:

generate a plurality of frequency domain bins from a block; and

determine whether the block is valid by performing an energy analysis in a subset of the plurality of frequency domain bins.

12 . The image capture device of claim 11 , wherein the processor is further configured to:

determine that the block is valid if, for a W channel in the Ambisonics signals, a total energy across the subset of the plurality of frequency domain bins exceeds an active signal threshold.

13 . The image capture device of claim 11 , wherein the processor is further configured to:

determine, for an X channel in the Ambisonics signals, an X signal threshold based on an X channel energy contribution to a W channel energy;

determine, for a Y channel in the Ambisonics signals, a Y signal threshold based on an Y channel energy contribution to the W channel energy;

determine, for a Z channel in the Ambisonics signals, a Z signal threshold based on an Z channel energy contribution to the W channel energy; and

designate a frequency domain bin in the subset of the plurality of frequency domain bins as a key frequency domain bin if at least one of an X channel energy value exceeds the X signal threshold, a Y channel energy value exceeds the Y signal threshold, or a Z channel energy value exceeds the Z signal threshold.

14 . The image capture device of claim 11 , wherein the processor is further configured to:

generate a plurality of frequency domain bins from a block; and

determine that the block is valid if, for a W channel in the Ambisonics signals, a total energy across a subset of the plurality of frequency domain bins exceeds an active signal threshold.

15 . The image capture device of claim 11 , wherein the plurality of azimuth segments defines a 360° azimuth space.

16 . The image capture device of claim 11 , wherein the plurality of elevation segments defines an elevation space between +90° to −90°.

17 . The image capture device of claim 11 , wherein the processor is further configured to:

forego application of rotation parameters to the Ambisonics signals prior to determination of the azimuth angle estimate and the elevation angle estimate when processing is performed on the image capture device.

18 . A method comprising:

determining a direction of arrival of Ambisonics signals for a time interval when a defined number of blocks in the time interval are valid, wherein the determining is further comprising:

determining an azimuth angle estimate and an elevation angle estimate for the time interval;

generating the azimuth angle estimate based on a maximum number of azimuth angle estimates present in an azimuth segment from a number of azimuth segments defining an azimuth space;

generating the elevation angle estimate based a maximum number of elevation angle estimate present in an elevation segment from a number of elevation segments defining an elevation space;

determining, for the azimuth angle estimate, an azimuth angle estimate confidence level based on a ratio of the maximum number of azimuth angle estimates present in the azimuth segment and a total number of azimuth angle estimates present in the number of azimuth segments; and

determining, for the elevation angle estimate, an elevation angle estimate confidence level based on a ratio between the maximum number of elevation angle estimates present in the elevation segment and a total number of elevation angle estimates present in the number of elevation segments,

wherein the direction of arrival is represented by the azimuth angle estimate and the elevation angle estimate.

19 . The method of claim 18 , wherein the determining the azimuth angle estimate and the elevation angle estimate for the time interval is further comprising:

performing, in a subset of frequency domain bins from a number of frequency domain bins generated from a block of the defined number of blocks, an energy analysis to determine whether the block is valid.

20 . The method of claim 19 , wherein a frequency domain bin has a spectral value which is representative of an energy level in the frequency domain bin.

Assignments (3)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2023
From: MESIWALA, HAKIM; PENROD, ERIC STEVEN; TISCH, ERICH; HARDIN, MARK; DICK, TIMOTHY; MACDONALD, WILLIAM EDWARD
To: GOPRO, INC.
Reel/Frame 063839/0824 →
Continuity (1)
Related Publication 20240406652A1 · Dec 5, 2024
References Cited (5)
US 9769588B2 · Shenoy · 2017 [cited by examiner]
US 20200154229A1 · Habets · 2020 [cited by examiner]
US 20200275227A1 · Habets · 2020 [cited by examiner]
WO WO2022108494A1 · 2022 [cited by examiner]
Khaddour, Hasan, Jiri Schimmel, and Michal Trzos. “Estimation of direction of arrival of multiple sound sources in 3d space using b-format.” International Journal of Advances in Telecommunications, Electrotechnics, Sign… [cited by applicant]