IP Library Granted Patent US 10,657,974
Granted Patent B2
US 10,657,974 · App. 16/227,880 · Granted May 19, 2020

Priority information for higher order ambisonic audio data

Inventors: Moo Young Kim (San Diego, CA); Nils Günther Peters (San Diego, CA); Shankar Thagadur Shivappa (San Diego, CA); Dipanjan Sen (San Diego, CA)
Assignee: Qualcomm Incorporated
G10L19/008G10L19/167H04S7/30H04S3/008H04S2400/11H04S2400/15H04S2420/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,657,974
App. No.
16/227,880
Granted
May 19, 2020
Kind
B2
Abstract

In general, techniques are described by which to provide priority information for higher order ambisonic (HOA) audio data. A device comprising a memory and a processor may perform the techniques. The memory stores HOA coefficients of the HOA audio data, the HOA coefficients representative of a soundfield. The processor may decompose the HOA coefficients into a sound component and a corresponding spatial component, the corresponding spatial component defining shape, width, and directions of the sound component, and the corresponding spatial component defined in a spherical harmonic domain. The processor may also determine, based on one or more of the sound component and the corresponding spatial component, priority information indicative of a priority of the sound component relative to other sound components of the soundfield, and specify, in a data object representative of a compressed version of the HOA audio data, the sound component and the priority information.

Claims (83)

1. A device configured to compress higher order ambisonic audio data representative of a soundfield, the device comprising:

a memory configured to store higher order ambisonic coefficients of the higher order ambisonic audio data, the higher order ambisonic coefficients representative of a soundfield; and

one or more processors configured to:

decompose the higher order ambisonic coefficients into a sound component and a corresponding spatial component, the corresponding spatial component defining shape, width, and directions of the sound component in a spherical harmonic domain;

determine, based on one or more of the sound component and the corresponding spatial component, priority information indicative of a priority of the sound component relative to other sound components of the soundfield; and

specify, in a data object representative of a compressed version of the higher order ambisonic audio data, the sound component and the priority information.

2. The device of claim 1 ,

wherein the one or more processors are further configured to obtain, based on the sound component and the corresponding spatial component, a higher order ambisonic representation of the sound component, and

wherein the one or more processors are configured to determine, based on one or more of the higher order ambisonic representation of the sound component and the corresponding spatial component, the priority information.

3. The device of claim 2 , wherein the one or more processors are configured to:

render the higher order ambisonic representation of the sound component to one or more speaker feeds; and

wherein the one or more processors are configured to determine, based on one or more of the higher order ambisonic representation of the sound component, the speaker feeds, and the corresponding spatial component, the priority information.

4. The device of claim 1 , wherein the one or more processors are configured to:

determine, based on the corresponding spatial component, a spatial weighting indicative of a relevance of the sound component to the soundfield; and

determine, based on one or more of the sound component, the higher order ambisonic representation of the sound component, the one or more speaker feeds, and the spatial weighting, the priority information.

5. The device of claim 1 , wherein the one or more processors are configured to:

determine an energy associated with the sound component, the higher order ambisonic representation of the sound component, or the one or more speaker feeds; and

determine, based on one or more of the energy and the spatial weighting, the priority information.

6. The device of claim 1 , wherein the one or more processors are configured to:

determine a loudness measure associated with one of the sound component, the higher order ambisonic representation of the sound component, or the one or more speaker feeds, the loudness measure indicative of a relevance of the sound component to the soundfield;

determine, based on one or more of the loudness measure and the spatial weighting, the priority information.

7. The device of claim 1 , wherein the one or more processors are configured to:

determining continuity indication indicative of whether a current portion defines the same sound component as a previous portion of the data object;

determine, based on one or more of the continuity indication and the spatial weighting, the priority information.

8. The device of claim 1 , wherein the one or more processors are configured to:

perform signal classification with respect to the sound component, the higher order ambisonic representation of the sound component, or the one or more speaker feeds to determine a class to which the sound component corresponds;

determine, based on one or more of the class and the spatial weighting, the priority information.

9. The device of claim 8 , wherein the one or more processors are configured to perform signal classification with respect to the sound component, the higher order ambisonic representation of the sound component, or the one or more speaker feeds to determine a speech class or a non-speech class to which the sound component corresponds.

10. The device of claim 1 ,

wherein the data object comprises a bitstream,

wherein the bitstream comprises a plurality of transport channels,

wherein the priority information comprises priority channel information, and

wherein the one or more processors are configured to:

specify, in a transport channel of the plurality of transport channels, the sound component; and

specify, in the bitstream, the priority channel information indicative of a priority of the transport channel relative to remaining ones of the plurality of transport channels defining the other sound components.

11. The device of claim 1 ,

wherein the data object comprises a file,

wherein the file comprises a plurality of tracks,

wherein the priority information comprises priority track information, and

wherein the one or more processors are configured to:

specify, in a track of the plurality of tracks, the sound component; and

specify, in the bitstream, the priority track information indicative of a priority of the track relative to remaining ones of the plurality of tracks defining the other sound components.

12. The device of claim 1 , wherein the one or more processors are configured to:

receive the higher order ambisonic audio data; and

output the data object to an emission encoder, the emission encoder configured to transcode the bitstream based on a target bitrate.

13. The device of claim 1 , further comprising a microphone configured to capture spatial audio data representative of the higher order ambisonic audio data, and convert the spatial audio data to the higher order ambisonic audio data.

14. The device of claim 1 , wherein the device comprises a robotic device.

15. The device of claim 1 , wherein the device comprises a flying device.

16. A method of compressing higher order ambisonic audio data representative of a soundfield, the method comprising:

decomposing higher order ambisonic coefficients of the ambisonic higher order ambisonic audio data into a sound component and a corresponding spatial component, the higher order ambisonic audio data representative of a soundfield, the corresponding spatial component defining shape, width, and directions of the sound component, and the corresponding spatial component defined in a spherical harmonic domain;

determining, based on one or more of the sound component and the corresponding spatial component, priority information indicative of a priority of the sound component relative to other sound components of the soundfield; and

specifying, in a data object representative of a compressed version of the higher order ambisonic audio data, the sound component and the priority information.

17. The method of claim 16 , wherein determining the priority information comprises:

obtaining, from a content provider providing the higher order ambisonic audio data, a preferred priority of the sound component relative to other sound components of the soundfield; and

determining, based on one or more of the preferred priority and the spatial weighting, the priority information.

18. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the energy, the continuity indication, and the spatial weighting, the priority information.

19. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the loudness measure, the continuity indication, and the spatial weighting, the priority information.

20. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the energy, the class, and the spatial weighting, the priority information.

21. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the loudness measure, the class, and the spatial weighting, the priority information.

22. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the energy, the preferred priority, and the spatial weighting, the priority information.

23. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the loudness measure, the preferred priority, and the spatial weighting, the priority information.

24. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the energy, the continuity indication, the class, the preferred priority, and the spatial weighting, the priority information.

25. The method of claim 16 , wherein determining the priority information comprises determining, based on one or more of the loudness measure the continuity indication, the class, the preferred priority, and the spatial weighting, the priority information.

26. The method of claim 16 ,

wherein the data object comprises a bitstream,

wherein the bitstream comprises a plurality of transport channels,

wherein the priority information comprises priority channel information, and

wherein specifying the sound component comprises specifying, in a transport channel of the plurality of transport channels, the sound component; and

wherein specifying the priority information comprises specifying, in the bitstream, the priority channel information indicative of a priority of the transport channel relative to remaining ones of the plurality of transport channels defining the other sound components.

27. The method of claim 16 ,

wherein the data object comprises a file,

wherein the file comprises a plurality of tracks,

wherein the priority information comprises priority track information,

wherein specifying the sound component comprises specifying, in a track of the plurality of tracks, the sound component, and

wherein specifying the priority information comprises specifying, in the bitstream, the priority track information indicative of a priority of the track relative to remaining ones of the plurality of tracks defining the other sound components.

28. The method of claim 16 , further comprising:

receiving the higher order ambisonic audio data; and

outputting the data object to an emission encoder, the emission encoder configured to transcode the bitstream based on a target bitrate.

29. The method of claim 16 , further comprising capturing, by a microphone, spatial audio data representative of the higher order ambisonic audio data, and convert the spatial audio data to the higher order ambisonic audio data.

30. A device configured to compress higher order ambisonic audio data representative of a soundfield, the device comprising:

means for decomposing higher order ambisonic coefficients of the ambisonic higher order ambisonic audio data into a sound component and a corresponding spatial component, the higher order ambisonic audio data representative of a soundfield, the corresponding spatial component defining shape, width, and directions of the sound component, and the corresponding spatial component defined in a spherical harmonic domain;

means for determining, based on one or more of the sound component and the corresponding spatial component, priority information indicative of a priority of the sound component relative to other sound components of the soundfield; and

means for specifying, in a data object representative of a compressed version of the higher order ambisonic audio data, the sound component and the priority information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 2, 2019
From: KIM, MOO YOUNG; PETERS, NILS GUNTHER; THAGADUR SHIVAPPA, SHANKAR; SEN, DIPANJAN
To: QUALCOMM INCORPORATED
Reel/Frame 048771/0970 →
Continuity (2)
Provisional Application 62609157 · Dec 21, 2017
Related Publication 20190198028A1 · Jun 27, 2019
Cited By (1)
US 12,308,034