IP Library Granted Patent US 11,315,578
Granted Patent B2
US 11,315,578 · App. 17/047,403 · Granted Apr 26, 2022

Methods, apparatus and systems for encoding and decoding of directional sound sources

Inventors: Nicolas R. Tsingos (San Francisco, CA); Mark R. P. Thomas (Walnut Creek, CA); Christof Fersch (Neumarkt, DE)
Assignees: Dolby Laboratories Licensing Corporation; Dolby International AB
G10L19/008H04S7/302H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,315,578
App. No.
17/047,403
Granted
Apr 26, 2022
Kind
B2
Abstract

Some disclosed methods involve encoding or decoding directional audio data. Some encoding methods may involve receiving a mono audio signal corresponding to an audio object and a representation of a radiation pattern corresponding to the audio object. The radiation pattern may include sound levels corresponding to plurality of sample times, a plurality of frequency bands and a plurality of directions. The methods may involve encoding the mono audio signal and encoding the source radiation pattern to determine radiation pattern metadata. Encoding the radiation pattern may involve determining a spherical harmonic transform of the representation of the radiation pattern and compressing the spherical harmonic transform to obtain encoded radiation pattern metadata.

Claims (36)

1. A method for encoding directional audio data, comprising:

receiving a mono audio signal corresponding to an audio object and a representation of a radiation pattern corresponding to the audio object, the radiation pattern comprising sound levels corresponding to plurality of sample times, a plurality of frequency bands and a plurality of directions;

encoding the mono audio signal;

resealing the radiation pattern to an amplitude of the radiation pattern in a particular direction on a per frequency basis to determine a normalized radiation pattern; and

encoding the normalized radiation pattern to determine radiation pattern metadata,

wherein the encoding of the normalized radiation pattern comprises determining a spherical harmonic transform of the representation of the normalized radiation pattern and compressing the spherical harmonic transform to obtain encoded radiation pattern metadata corresponding to the normalized radiation pattern.

2. The method of claim 1 , further comprising encoding a plurality of directional audio objects based on a cluster of audio objects, wherein the radiation pattern is representative of a centroid that reflects an average sound level value for each frequency band.

3. The method of claim 2 , wherein the plurality of directional audio objects is encoded as a single directional audio object whose directivity corresponds with a time-varying energy-weighted average of each audio object's spherical harmonic coefficients, and/or wherein the encoded radiation pattern metadata indicates a position of a cluster of audio objects that is an average of corresponding positions of the plurality of directional audio objects.

4. The method of claim 1 , further comprising encoding group metadata regarding a radiation pattern of a group of directional audio objects.

5. The method of claim 1 , wherein compressing the spherical harmonic transform comprises at least one of a Singular Value Decomposition method, principal component analysis, discrete cosine transforms, data-independent bases, or eliminating spherical harmonic coefficients of the spherical harmonic transform that are above a threshold order of spherical harmonic coefficients.

6. A method for decoding audio data, comprising:

receiving an encoded core audio signal, encoded radiation pattern metadata and encoded audio object metadata, wherein the audio object metadata includes at least one of time-varying 3 degrees of freedom (DoF) or 6DoF source orientation information;

decoding the encoded core audio signal to determine a core audio signal;

decoding the encoded radiation pattern metadata to determine a decoded radiation pattern;

decoding the audio object metadata; and

rendering the core audio signal based on the audio object metadata and the decoded radiation pattern.

7. The method of claim 6 , wherein the core audio signal comprises a plurality of directional objects based on a cluster of objects, and wherein the decoded radiation pattern is representative of a centroid that reflects an average value for each frequency band.

8. The method of claim 6 , wherein the encoded radiation pattern metadata corresponds with a time- and frequency-varying set of spherical harmonic coefficients.

9. The method of claim 6 , wherein the encoded radiation pattern metadata comprises audio object type metadata.

10. The method of claim 9 , wherein the audio object type metadata indicates parametric directivity pattern data and wherein the parametric directivity pattern data includes one or more functions selected from a list of functions that consists of a cosine function, a sine function or a cardioidal function.

11. The method of claim 9 , wherein the audio object type metadata indicates dynamic directivity pattern data and wherein the dynamic directivity pattern data corresponds with a time- and frequency-varying set of spherical harmonic coefficients.

12. The method of claim 11 , further comprising receiving the dynamic directivity pattern data prior to receiving the encoded core audio signal.

13. The method of claim 6 , wherein the rendering is based on applying subband gains, based at least in part on the decoded radiation pattern, to the decoded core audio signal.

14. The method of claim 9 wherein the audio object type metadata indicates database directivity pattern data and wherein decoding the encoded radiation pattern metadata to determine the decoded radiation pattern comprises querying a directivity data structure that includes audio object types and corresponding directivity pattern data.

15. An audio decoding apparatus, comprising:

an interface system; and

a control system configured for:

receiving, via the interface system, audio data corresponding to at least one audio object, the audio data including a monophonic audio signal, audio object position metadata, audio object size metadata, and a rendering parameter, wherein the audio object position metadata includes at least one of time-varying 3 degrees of freedom (DoF) or 6DoF source orientation information;

determining whether the rendering parameter indicates a positional mode or a directivity mode; and, upon determining that the rendering parameter indicates a directivity mode, rendering the audio data for reproduction via at least one loudspeaker according to a directivity pattern indicated by at least one of the audio object position metadata or the audio object size metadata.

16. The apparatus of claim 15 , wherein rendering the audio data comprises interpreting the audio object position metadata as audio object orientation metadata.

17. The apparatus of claim 16 , wherein the audio object position metadata comprises at least one of x,y,z coordinate data, spherical coordinate data or cylindrical coordinate data and wherein the audio object orientation metadata comprises yaw, pitch and roll data.

18. The apparatus of claim 15 , wherein rendering the audio data comprises interpreting the audio object size metadata as directivity metadata that corresponds to the directivity pattern.

19. The apparatus of claim 15 , wherein rendering the audio data comprises querying a data structure that includes a plurality of directivity patterns and mapping at least one of the audio object position metadata or the audio object size metadata to one or more of the directivity patterns.

20. The apparatus of claim 19 , wherein the control system is configured for receiving, via the interface system, the data structure.

21. The apparatus of claim 20 , wherein the data structure is received prior to the audio data.

22. The apparatus of claim 15 , wherein the audio data is received in a Dolby Atmos format, and/or wherein the audio object position metadata corresponds to world coordinates or model coordinates.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2020
From: TSINGOS, NICOLAS R.; THOMAS, MARK R.P.; FERSCH, CHRISTOF
To: DOLBY LABORATORIES LICENSING CORPORATION; DOLBY INTERNATIONAL AB
Reel/Frame 054071/0558 →
Continuity (4)
Provisional Application 62741419 · Oct 4, 2018
Provisional Application 62681429 · Jun 6, 2018
Provisional Application 62658067 · Apr 16, 2018
Related Publication 20210118452A1 · Apr 22, 2021