IP Library › Granted Patent US 10,841,726
Granted Patent B2
US 10,841,726 · App. 16/487,882 · Granted Nov 17, 2020

Immersive audio rendering

Inventor: Sunil Bharitkar (Palo Alto, CA)
Assignee: Hewlett-Packard Development Company, L.P.
H04S7/304H04R3/04H04R3/12H04R5/033H04R5/04H04S2400/01H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,841,726
App. No.
16/487,882
Granted
Nov 17, 2020
Kind
B2
Abstract

In some examples, immersive audio rendering may include determining whether an audio signal includes a first content format including stereo content, or a second content format including multichannel or object-based content. In response to a determination that the audio signal includes the first content format, the audio signal may be routed to a first block that includes a low-frequency extension and a stereo to multichannel upmix to generate a resulting audio signal. Alternatively, the audio signal may be routed to another low-frequency extension to generate the resulting audio signal. The audio signal may be further processed by performing spatial synthesis on the resulting audio signal, and crosstalk cancellation on the spatial synthesized audio signal. Further, multiband-range compression may be performed on the crosstalk cancelled audio signal, and an output stereo signal may be generated based on the multiband-range compressed audio signal.

Claims (62)

1. An apparatus comprising:

a processor; and

a non-transitory computer readable medium storing machine readable instructions that when executed by the processor cause the processor to:

determine whether an audio signal includes a first content format including stereo content, or a second content format including multichannel or object-based content;

in response to a determination that the audio signal includes the first content format, route the audio signal to a low-frequency extension and a stereo to multichannel upmix to generate a resulting audio signal;

in response to a determination that the audio signal includes the second content format, route the audio signal to another low-frequency extension to generate the resulting audio signal;

perform spatial synthesis on the resulting audio signal;

perform crosstalk cancellation on the spatial synthesized audio signal;

perform multiband-range compression on the crosstalk cancelled audio signal; and

generate an output stereo signal based on the multiband-range compressed audio signal.

2. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

route the audio signal, in response to the determination that the audio signal includes the first content format, to the low-frequency extension to

band pass filter non-linear terms in a plurality of filterbanks associated with the low-frequency extension, and

determine a sum of the band pass filtered non-linear terms to generate low frequencies associated with the audio signal.

3. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

route the audio signal, in response to the determination that the audio signal includes the second content format, to the another low-frequency extension to

band pass filter non-linear terms in a plurality of filterbanks associated with the another low-frequency extension, and

determine a sum of the band pass filtered non-linear terms to generate low frequencies associated with the audio signal.

4. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

route the audio signal, in response to the determination that the audio signal includes the first content format, to the stereo to multichannel upmix to

increase a number of channels associated with the audio signal.

5. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

perform spatial synthesis on the resulting audio signal by applying head-related transfer functions (HRTFs) to render virtual sources associated with the resulting audio signal at specified angles; and

reduce a number of channels associated with the virtual sources by performing a linear summation operation with respect to each of the channels.

6. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

perform crosstalk cancellation on the spatial synthesized audio signal by performing equalization of ipsilateral signals associated with the audio signal and cancellation of contralateral crosstalk associated with the audio signal.

7. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

perform multiband-range compression on the crosstalk cancelled audio signal by using perfect reconstruction filterbanks.

8. The apparatus according to claim 1 , wherein the instructions are further to cause the processor to:

perform decorrelation on the spatial synthesized audio signal by implementing complementary filters as a function of z, N, and λ, where z=e{circumflex over ( )}{jw}, j=sqrt(−1), w represents angular frequency, N represents an integer that determines an order of an all-pass filter, and λ represents a coefficient, where |λ|<1.

9. A method comprising:

determining, by a processor, whether an audio signal includes a first content format including stereo content, or a second content format including multichannel or object-based content;

in response to a determination that the audio signal includes the first content format, routing the audio signal to a low-frequency extension and a stereo to multichannel upmix to generate a resulting audio signal, wherein the stereo to multichannel upmix is to increase a number of channels associated with the audio signal;

in response to a determination that the audio signal includes the second content format, routing the audio signal to another low-frequency extension to generate the resulting audio signal;

performing spatial synthesis on the resulting audio signal;

performing crosstalk cancellation on the spatial synthesized audio signal;

performing multiband-range compression on the crosstalk cancelled audio signal; and

generating an output stereo signal based on the multiband-range compressed audio signal.

10. The method according to claim 9 , further comprising:

routing the audio signal, in response to the determination that the audio signal includes the first content format, to the low-frequency extension to

band pass filter non-linear terms in a plurality of filterbanks associated with the low-frequency extension, and

determine a sum of the band pass filtered non-linear terms to generate low frequencies associated with the audio signal.

11. The method according to claim 9 , further comprising:

routing the audio signal, in response to the determination that the audio signal includes the second content format, to the another low-frequency extension to

band pass filter non-linear terms in a plurality of filterbanks associated with the another low-frequency extension, and

determine a sum of the band pass filtered non-linear terms to generate low frequencies associated with the audio signal.

12. A non-transitory computer readable medium having stored thereon machine readable instructions, the machine readable instructions, when executed, cause a processor to:

determine whether an audio signal includes a first content format including stereo content, or a second content format including multichannel or object-based content;

in response to a determination that the audio signal includes the first content format, route the audio signal to a low-frequency extension and a stereo to multichannel upmix to generate a resulting audio signal;

in response to a determination that the audio signal includes the second content format, route the audio signal to another low-frequency extension to generate the resulting audio signal;

perform spatial synthesis on the resulting audio signal;

perform crosstalk cancellation on the spatial synthesized audio signal;

perform multiband-range compression on the crosstalk cancelled audio signal by using perfect reconstruction filterbanks; and

generate an output stereo signal based on the multiband-range compressed audio signal.

13. The non-transitory computer readable medium according to claim 12 , wherein the instructions are further to cause the processor to:

route the audio signal, in response to the determination that the audio signal includes the first content format, to the stereo to multichannel upmix to

increase a number of channels associated with the audio signal.

14. The non-transitory computer readable medium according to claim 12 , wherein the instructions are further to cause the processor to:

perform spatial synthesis on the resulting audio signal by applying head-related transfer functions (HRTFs) to render virtual sources associated with the resulting audio signal at specified angles; and

reduce a number of channels associated with the virtual sources by performing a linear summation operation with respect to each of the channels.

15. The non-transitory computer readable medium according to claim 12 , wherein the instructions are further to cause the processor to:

perform decorrelation on the spatial synthesized audio signal by implementing complementary filters as a function of z, N, and λ, where z=e{circumflex over ( )}{jw}, j=sqrt(−1), w represents angular frequency, N represents an integer that determines an order of an all-pass filter, and λ represents a coefficient, where |λ|<1.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2019
From: BHARITKAR, SUNIL
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 050128/0783 →
Continuity (1)
Related Publication 20200236488A1 · Jul 23, 2020