IP Library Granted Patent US 11,361,776
Granted Patent B2
US 11,361,776 · App. 16/907,969 · Granted Jun 14, 2022

Coding scaled spatial components

Inventors: Ferdinando Olivieri (San Diego, CA); Taher Shahbazi Mirzahasanloo (San Diego, CA); Nils Günther Peters (San Diego, CA)
Assignee: Qualcomm Incorporated
G10L19/008G10L19/032H04S7/30H04S2420/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,776
App. No.
16/907,969
Granted
Jun 14, 2022
Kind
B2
Abstract

In general, techniques are described by which to code scaled spatial components. A device comprising a memory and one or more processors may be configured to perform the techniques. The memory may store a bitstream including an encoded foreground audio signal and a corresponding quantized spatial component. The one or more processors may perform psychoacoustic audio decoding with respect to the encoded foreground audio signal to obtain a foreground audio signal, and determine, when performing psychoacoustic audio decoding, a bit allocation for the encoded foreground audio signal. The one or more processors may dequantize the quantized spatial component to obtain a scaled spatial component, and descale, based on the bit allocation, the scaled spatial component to obtain a spatial component. The one or more processors may reconstruct, based on the foreground audio signal and the spatial component, scene-based audio data.

Claims (74)

1. A device configured to encode scene-based audio data, the device comprising:

a memory configured to store the scene-based audio data; and

one or more processors configured to:

perform spatial audio encoding with respect to the scene-based audio data to obtain a foreground audio signal and a corresponding spatial component, the spatial component defining spatial characteristics of the foreground audio signal;

perform psychoacoustic audio encoding with respect to the foreground audio signal to obtain an encoded foreground audio signal;

determine, when performing psychoacoustic audio encoding with respect to the foreground audio signal, a bit allocation for the foreground audio signal;

scale, based on the bit allocation for the foreground audio signal, the spatial component to obtain a scaled spatial component;

quantize the scaled spatial component to obtain a quantized spatial component; and

specify, in a bitstream, the encoded foreground audio signal and the quantized spatial component.

2. The device of claim 1 , wherein the one or more processors are configured to perform psychoacoustic audio encoding according to a compression algorithm with respect to the foreground audio signal to obtain the encoded foreground audio signal.

3. The device of claim 1 , wherein the one or more processors are configured to:

perform a shape and gain analysis with respect to the foreground audio signal to obtain a shape and a gain representative of the foreground audio signal;

perform quantization with respect to the gain to obtain a course quantized gain and one or more fine quantized residuals; and

scale, based on a number of bits allocated to the course quantized gain and each of the one or more fine quantized residuals, the spatial component to obtain the scaled spatial component.

4. The device of claim 1 , wherein the one or more processors are configured to perform a linear invertible transform with respect to the scene-based audio data to obtain the foreground audio signal and the corresponding spatial component.

5. The device of claim 1 , wherein the scene-based audio data comprises ambisonic coefficients corresponding to an order greater than one.

6. The device of claim 1 , wherein the scene-based audio data comprises ambisonic coefficients corresponding to an order greater than zero.

7. The device of claim 1 , wherein the scene-based audio data comprises audio data defined in a spherical harmonic domain.

8. The device of claim 1 ,

wherein the foreground audio signal comprises a foreground audio signal defined in the spherical harmonic domain, and

wherein the spatial component comprises a spatial component defined in the spherical harmonic domain.

9. The device of claim 1 , wherein the scene-based audio data comprises mixed-order ambisonic audio data.

10. A method of encoding scene-based audio data, the method comprising:

performing spatial audio encoding with respect to the scene-based audio data to obtain a foreground audio signal and a corresponding spatial component, the spatial component defining spatial characteristics of the foreground audio signal;

performing psychoacoustic audio encoding with respect to the foreground audio signal to obtain an encoded foreground audio signal;

determining, when performing psychoacoustic audio encoding with respect to the foreground audio signal, a bit allocation for the foreground audio signal;

scaling, based on the bit allocation for the foreground audio signal, the spatial component to obtain a scaled spatial component;

quantizing the scaled spatial component to obtain a quantized spatial component; and

specifying, in a bitstream, the encoded foreground audio signal and the quantized spatial component.

11. A device configured to decode a bitstream representative of encoded scene-based audio data, the device comprising:

a memory configured to store the bitstream, the bitstream including an encoded foreground audio signal and a corresponding quantized spatial component that defines spatial characteristics of the encoded foreground audio signal; and

one or more processors configured to:

perform psychoacoustic audio decoding with respect to the encoded foreground audio signal to obtain a foreground audio signal;

determine, when performing psychoacoustic audio decoding with respect to the encoded foreground audio signal, a bit allocation for the encoded foreground audio signal;

dequantize the quantized spatial component to obtain a scaled spatial component;

descale, based on the bit allocation for the encoded foreground audio signal, the scaled spatial component to obtain a spatial component; and

reconstruct, based on the foreground audio signal and the spatial component, the scene-based audio data.

12. The device of claim 11 , wherein the one or more processors are configured to perform psychoacoustic audio decoding according to an AptX compression algorithm with respect to the encoded foreground audio signal to obtain the foreground audio signal.

13. The device of claim 11 , wherein the one or more processors are configured to:

obtain, from the bitstream, a number of bits allocated to a course quantized gain and each of one or more fine quantized residuals, the course quantized gain and the one or more fine quantized residual represent a gain of the foreground audio signal; and descale, based on the number of bits allocated to the course quantized gain and each of the one or more fine quantized residuals, the scaled spatial component to obtain the spatial component.

14. The device of claim 11 , wherein the scene-based audio data comprises ambisonic coefficients corresponding to an order greater than one.

15. The device of claim 11 , wherein the scene-based audio data comprises ambisonic coefficients corresponding to an order greater than zero.

16. The device of claim 11 , wherein the scene-based audio data comprises audio data defined in a spherical harmonic domain.

17. The device of claim 11 ,

wherein the encoded foreground audio signal comprises an encoded foreground audio signal defined in the spherical harmonic domain, and

wherein the scaled spatial component comprises a scaled spatial component defined in the spherical harmonic domain.

18. The device of claim 11 , wherein the one or more processors are further configured to:

render the scene-based audio data to one or more speaker feeds; and

reproduce, based on the speaker feeds, a soundfield represented by the scene-based audio data.

19. The device of claim 11 ,

wherein the one or more processors are further configured to render the scene-based audio data to one or more speaker feeds, and

wherein the device comprises one or more speakers configured to reproduce, based on the speaker feeds, a soundfield represented by the scene-based audio data.

20. The device of claim 11 , wherein the scene-based audio data comprises mixed-order ambisonic audio data.

21. A method of decoding a bitstream representative of scene-based audio data, the method comprising:

obtaining, from the bitstream, an encoded foreground audio signal and a corresponding quantized spatial component that defines the spatial characteristics of the encoded foreground audio signal;

performing psychoacoustic audio decoding with respect to the encoded foreground audio signal to obtain a foreground audio signal;

determining, when performing psychoacoustic audio decoding with respect to the encoded foreground audio signal, a bit allocation for the encoded foreground audio signal;

dequantizing the quantized spatial component to obtain a scaled spatial component;

descaling, based on the bit allocation for the encoded foreground audio signal, the scaled spatial component to obtain a spatial component; and

reconstructing, based on the foreground audio signal and the spatial component, the scene-based audio data.

22. The method of claim 21 , wherein performing psychoacoustic audio decoding comprises performing psychoacoustic audio decoding according to a compression algorithm with respect to the encoded foreground audio signal to obtain the foreground audio signal.

23. The method of claim 21 ,

wherein determining the bit allocation comprises obtaining, from the bitstream, a number of bits allocated to a course quantized gain and each of one or more fine quantized residuals, the course quantized gain and the one or more fine quantized residual represent a gain of the foreground audio signal, and

wherein descaling the scaled spatial component comprises descaling, based on the number of bits allocated to the course quantized gain and each of the one or more fine quantized residuals, the scaled spatial component to obtain the spatial component.

24. The method of claim 21 , wherein the scene-based audio data includes ambisonic coefficients corresponding to a spherical basis function having an order greater than zero.

25. The method of claim 21 , wherein the scene-based audio data comprises higher order ambisonic coefficients corresponding to an order greater than one.

26. The method of claim 21 , wherein the scene-based audio data comprises audio data defined in a spherical harmonic domain.

27. The method of claim 21 ,

wherein the encoded foreground audio signal comprises an encoded foreground audio signal defined in the spherical harmonic domain, and

wherein the scaled spatial component comprises a scaled spatial component defined in the spherical harmonic domain.

28. The method of claim 21 , further comprising:

rendering the scene-based audio data to one or more speaker feeds; and

reproducing, based on the speaker feeds, a soundfield represented by the scene-based audio data.

29. The method of claim 21 , wherein the scene-based audio data comprises mixed-order ambisonic audio data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 21, 2020
From: OLIVIERI, FERDINANDO; SHAHBAZI MIRZAHASANLOO, TAHER; PETERS, NILS GÜNTHER
To: QUALCOMM INCORPORATED
Reel/Frame 054711/0413 →
Continuity (2)
Provisional Application 62865858 · Jun 24, 2019
Related Publication 20200402519A1 · Dec 24, 2020