IP Library Granted Patent US 10,492,018
Granted Patent B1
US 10,492,018 · App. 15/290,717 · Granted Nov 26, 2019

Symmetric binaural rendering for high-order ambisonics

Inventor: Andrew Allen (San Jose, CA)
Assignee: GOOGLE LLC
H04S7/304H04R5/033H04S3/008H04S2400/11H04S2420/01H04S2420/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,492,018
App. No.
15/290,717
Granted
Nov 26, 2019
Kind
B1
Abstract

Techniques of performing binaural rendering involve combining left and right HRTFs from speakers symmetrically placed with respect to a forward axis of a listener's head. Because of the symmetry of the listening geometry, a decoded loudspeaker signal resulting from each speaker may be decomposed into symmetric and antisymmetric components according to whether a spherical harmonic associated with a respective ambisonic channel is symmetric or antisymmetric about the forward axis. The result involves rendering the left and right headphone speakers using the same sum and difference HRTFs rather than a different pair of HRFTs for each headphone speaker.

Claims (86)

1. A method, comprising:

receiving, by a sound rendering computer configured to render sound fields for presentation to a left ear and a right ear of a head of a human listener, sound data indicating a sound field that has (i) a first component that is symmetric about a forward axis of the head of the human listener and (ii) a second component that is antisymmetric about the forward axis, wherein the sound field is represented by a first virtual loudspeaker and a second virtual loudspeaker placed symmetrically about the forward axis, wherein receiving the sound data includes:

acquiring weights of ambisonic channels of the first virtual loudspeaker; and

forming a first set of the weights of the ambisonic channels corresponding to those ambisonic channels that are symmetric about the forward axis and a second set of the weights of the ambisonic channels corresponding to those ambisonic channels that are antisymmetric with respect to the forward axis;

acquiring a left head-related transfer function (HRTF) corresponding to the left ear and a right HRTF corresponding to the right ear;

generating (i) a difference HRTF as a difference between the left HRTF and the right HRTF and (ii) a sum HRTF as a sum of the left HRTF and the right HRTF;

performing (i) a first convolution operation on the first component of the sound field with the sum HRTF to produce a sum field, wherein performing the first convolution operation on the first component of the sound field with the sum HRTF includes convolving the first set of weights with the sum HRTF and (ii) a second convolution operation on the second component of the sound field with the difference HRTF to produce a difference field, wherein performing the second convolution operation on the second component of the sound field with the difference HRTF includes convolving the second set of weights with the sum HRTF; and

rendering the sound fields for presentation to the left ear and the right ear based on the sum field and the difference field.

2. The method as in claim 1 , wherein rendering the sound fields for presentation to the left ear and the right ear includes performing (i) a sum operation on the sum field and the difference field to render a left sound field in the left ear and (ii) a difference operation on the sum field and the difference field to render a right sound field in the right ear.

3. The method of claim 1 , wherein the generated difference HRTF is nonzero.

4. A method, comprising:

receiving, by a sound rendering computer configured to render sound fields for presentation to a left ear and a right ear of a head of a human listener, sound data indicating a sound field that has (i) a first component that is symmetric about a forward axis of the head of the human listener and (ii) a second component that is antisymmetric about the forward axis, wherein the sound field is emitted by a first loudspeaker and a second loudspeaker placed symmetrically about the forward axis,

wherein receiving the sound data includes:

acquiring gain coefficients of the first loudspeaker and the second loudspeaker, each of the gain coefficients of the first loudspeaker and each of the gain coefficients of the second loudspeaker corresponding to a spherical harmonic; and

from the gain coefficients of the first loudspeaker and the gain coefficients of the second loudspeaker, forming a first set of the gain coefficients corresponding to those spherical harmonics that are symmetric about the forward axis and a second set of the gain coefficients corresponding to those spherical harmonics that are antisymmetric with respect to the forward axis,

acquiring a left head-related transfer function (HRTF) corresponding to the left ear and a right HRTF corresponding to the right ear;

generating (i) a difference HRTF as a difference between the left HRTF and the right HRTF and (ii) a sum HRTF as a sum of the left HRTF and the right HRTF;

performing (i) a first convolution operation on the first component of the sound field with the sum HRTF to produce a sum field, wherein performing the first convolution operation on the first component of the sound field with the sum HRTF includes convolving the first set of gain coefficients with the sum HRTF and (ii) a second convolution operation on the second component of the sound field with the difference HRTF to produce a difference field, wherein performing the second convolution operation on the second component of the sound field with the difference HRTF includes convolving the second set of gain coefficients with the sum HRTF; and

rendering the sound field for presentation to the left ear and the right ear based on the sum field and the difference field.

5. The method as in claim 4 , wherein forming the first set of the gain coefficients includes, for each spherical harmonic, generating an average of the gain coefficient of the first loudspeaker corresponding to that spherical harmonic and the gain coefficient of the second loudspeaker corresponding to that spherical harmonic, and

wherein forming the second set of the gain coefficients includes, for each spherical harmonic, generating difference between the gain coefficient of the first loudspeaker corresponding to that spherical harmonic and the gain coefficient of the second loudspeaker corresponding to that spherical harmonic.

6. The method as in claim 4 , wherein the sound field is provided by a plurality of sound sources, and

wherein the method further comprises:

reading, from a memory of the sound rendering computer, a first net gain coefficient and a second net gain coefficient;

adding a gain coefficient of the first set of gain coefficients to the first net gain coefficient to produce a new first net gain coefficient;

adding a gain coefficient of the second set of gain coefficients to the second net gain coefficient to produce a new second net gain coefficient;

updating the first net gain coefficient with the new first net gain coefficient in the memory; and

updating the second net gain coefficient with the new second net gain coefficient in the memory.

7. A computer program product comprising a non-transitory storage medium, the storage medium including code that, when executed by processing circuitry of a sound rendering computer configured to render sound fields for presentation to a left ear and a right ear of a head of a human listener, causes the processing circuitry to perform a method, the method comprising:

receiving sound data indicating a sound field that has (i) a first component that is symmetric about a forward axis of the head of the human listener and (ii) a second component that is antisymmetric about the forward axis, wherein the sound field is represented by a first virtual loudspeaker and a second virtual loudspeaker placed symmetrically about the forward axis, wherein receiving the sound data includes:

acquiring weights of ambisonic channels of the first virtual loudspeaker; and

forming a first set of the weights of the ambisonic channels corresponding to those ambisonic channels that are symmetric about the forward axis and a second set of the weights of the ambisonic channels corresponding to those ambisonic channels that are antisymmetric with respect to the forward axis;

acquiring a left head-related transfer function (HRTF) corresponding to the left ear and a right HRTF corresponding to the right ear;

generating (i) a difference HRTF as a difference between the left HRTF and the right HRTF and (ii) a sum HRTF as a sum of the left HRTF and the right HRTF;

performing (i) a first convolution operation on the first component of the sound field with the sum HRTF to produce a sum, wherein performing the first convolution operation on the first component of the sound field with the sum HRTF includes convolving the first set of weights with the sum HRTF and (ii) a second convolution operation on the second component of the sound field with the difference HRTF to produce a difference field, wherein performing the second convolution operation on the second component of the sound field with the difference HRTF includes convolving the second set of weights with the sum HRTF; and

rendering the sound fields for presentation to the left ear and the right ear based on the sum field and the difference field.

8. The computer program product as in claim 7 , wherein rendering the sound fields for presentation to the left ear and the right ear include performing (i) a sum operation on the sum field and the difference field to render a left sound field in the left ear and (ii) a difference operation on the sum field and the difference field to render a right sound field in the right ear.

9. A computer program product comprising a non-transitory storage medium, the storage medium including code that, when executed by processing circuitry of a sound rendering computer configured to render sound fields for presentation to a left ear and a right ear of a head of a human listener, causes the processing circuitry to perform a method, the method comprising:

receiving sound data indicating a sound field that has (i) a first component that is symmetric about a forward axis of the head of the human listener and (ii) a second component that is antisymmetric about the forward axis, wherein the sound field is emitted by a first loudspeaker and a second loudspeaker placed symmetrically about the forward axis,

wherein receiving the sound data includes:

acquiring gain coefficients of the first loudspeaker and the second loudspeaker, each of the gain coefficients of the first loudspeaker and each of the gain coefficients of the second loudspeaker corresponding to a spherical harmonic; and

from the gain coefficients of the first loudspeaker and the gain coefficients of the second loudspeaker, forming a first set of the gain coefficients corresponding to those spherical harmonics that are symmetric about the forward axis and a second set of the gain coefficients corresponding to those spherical harmonics that are antisymmetric with respect to the forward axis,

acquiring a left head-related transfer function (HRTF) corresponding to the left ear and a right HRTF corresponding to the right ear;

generating (i) a difference HRTF as a difference between the left HRTF and the right HRTF and (ii) a sum HRTF as a sum of the left HRTF and the right HRTF;

performing (i) a first convolution operation on the first component of the sound field with the sum HRTF to produce a sum field, wherein performing the first convolution operation on the first component of the sound field with the sum HRTF includes convolving the first set of gain coefficients with the sum HRTF and (ii) a second convolution operation on the second component of the sound field with the difference HRTF to produce a difference field, wherein performing the second convolution operation on the second component of the sound field with the difference HRTF includes convolving the second set of gain coefficients with the sum HRTF; and

rendering the sound field for presentation to the left ear and the right ear based on the sum field and the difference field.

10. The computer program product as in claim 9 , wherein forming the first set of the gain coefficients includes, for each spherical harmonic, generating an average of the gain coefficient of the first loudspeaker corresponding to that spherical harmonic and the gain coefficient of the second loudspeaker corresponding to that spherical harmonic, and

wherein forming the second set of the gain coefficients includes, for each spherical harmonic, generating difference between the gain coefficient of the first loudspeaker corresponding to that spherical harmonic and the gain coefficient of the second loudspeaker corresponding to that spherical harmonic.

11. The computer program product as in claim 9 , wherein the sound field is provided by a plurality of sound sources, and

wherein the method further comprises:

reading, from a memory of the sound rendering computer, a first net gain coefficient and a second net gain coefficient;

adding a gain coefficient of the first set of gain coefficients to the first net gain coefficient to produce a new first net gain coefficient;

adding a gain coefficient of the second set of gain coefficients to the second net gain coefficient to produce a new second net gain coefficient;

updating the first net gain coefficient with the new first net gain coefficient in the memory; and

updating the second net gain coefficient with the new second net gain coefficient in the memory.

12. An electronic apparatus configured to render sound fields for presentation to a left ear and a right ear of a head of a human listener, the electronic apparatus comprising:

memory; and

controlling circuitry coupled to the memory, the controlling circuitry being configured to:

receive sound data indicating a sound field that has (i) a first component that is symmetric about a forward axis of the head of the human listener and (ii) a second component that is antisymmetric about the forward axis, wherein the sound field is represented by a first virtual loudspeaker and a second virtual loudspeaker placed symmetrically about the forward axis, wherein receiving the sound data includes:

acquiring weights of ambisonic channels of the first virtual loudspeaker; and

forming a first set of the weights of the ambisonic channels corresponding to those ambisonic channels that are symmetric about the forward axis and a second set of the weights of the ambisonic channels corresponding to those ambisonic channels that are anti symmetric with respect to the forward axis;

acquire a left head-related transfer function (HRTF) corresponding to the left ear and a right HRTF corresponding to the right ear;

generate (i) a difference HRTF as a difference between the left HRTF and the right HRTF and (ii) a sum HRTF as a sum of the left HRTF and the right HRTF;

perform (i) a first convolution operation on the first component of the sound field with the sum HRTF to produce a sum field, wherein performing the first convolution operation on the first component of the sound field with the sum HRTF includes convolving the first set of weights with the sum HRTF and (ii) a second convolution operation on the second component of the sound field with the difference HRTF to produce a difference field, wherein performing the second convolution operation on the second component of the sound field with the difference HRTF includes convolving the second set of weights with the sum HRTF; and

render the sound fields for presentation to the left ear and the right ear based on the sum field and the difference field.

13. The electronic apparatus as in claim 12 , wherein the controlling circuitry configured to render the sound fields for presentation to the left ear and the right ear is further configured to perform (i) a sum operation on the sum field and the difference field to render a left sound field in the left ear and (ii) a difference operation on the sum field and the difference field to render a right sound field in the right ear.

14. An electronic apparatus configured to render sound fields for presentation to a left ear and a right ear of a head of a human listener, the electronic apparatus comprising:

memory; and

controlling circuitry coupled to the memory, the controlling circuitry being configured to:

receive sound data indicating a sound field that has (i) a first component that is symmetric about a forward axis of the head of the human listener and (ii) a second component that is antisymmetric about the forward axis, wherein the sound field is emitted by a first loudspeaker and a second loudspeaker placed symmetrically about the forward axis,

wherein the controlling circuitry configured to receive the sound data is further configured to:

acquire gain coefficients of the first loudspeaker and the second loudspeaker, each of the gain coefficients of the first loudspeaker and each of the gain coefficients of the second loudspeaker corresponding to a spherical harmonic; and

from the gain coefficients of the first loudspeaker and the gain coefficients of the second loudspeaker, form a first set of the gain coefficients corresponding to those spherical harmonics that are symmetric about the forward axis and a second set of the gain coefficients corresponding to those spherical harmonics that are antisymmetric with respect to the forward axis,

acquire a left head-related transfer function (HRTF) corresponding to the left ear and a right HRTF corresponding to the right ear;

generate (i) a difference HRTF as a difference between the left HRTF and the right HRTF and (ii) a sum HRTF as a sum of the left HRTF and the right HRTF;

perform (i) a first convolution operation on the first component of the sound field with the sum HRTF to produce a sum field, wherein performing the first convolution operation on the first component of the sound field with the sum HRTF includes convolving the first set of gain coefficients with the sum HRTF and (ii) a second convolution operation on the second component of the sound field with the difference HRTF to produce a difference field, wherein performing the second convolution operation on the second component of the sound field with the difference HRTF includes convolving the second set of gain coefficients with the sum HRTF; and

render the sound field for presentation to the left ear and the right ear based on the sum field and the difference field.

15. The electronic apparatus as in claim 14 , wherein the controlling circuitry configured to form the first set of the gain coefficients is further configured to, for each spherical harmonic, generate an average of the gain coefficient of the first loudspeaker corresponding to that spherical harmonic and the gain coefficient of the second loudspeaker corresponding to that spherical harmonic, and

wherein the controlling circuitry configured to form the second set of the gain coefficients is further configured to, for each spherical harmonic, generate difference between the gain coefficient of the first loudspeaker corresponding to that spherical harmonic and the gain coefficient of the second loudspeaker corresponding to that spherical harmonic.

16. The electronic apparatus as in claim 14 , wherein the sound field is provided by a plurality of sound sources, and

wherein the controlling circuitry is further configured to:

read, from the memory, a first net gain coefficient and a second net gain coefficient;

add a gain coefficient of the first set of gain coefficients to the first net gain coefficient to produce a new first net gain coefficient;

add a gain coefficient of the second set of gain coefficients to the second net gain coefficient to produce a new second net gain coefficient;

update the first net gain coefficient with the new first net gain coefficient in the memory; and

update the second net gain coefficient with the new second net gain coefficient in the memory.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2016
From: ALLEN, ANDREW
To: GOOGLE INC.
Reel/Frame 039988/0636 →