IP Library › Granted Patent US 10,911,882
Granted Patent B2
US 10,911,882 · App. 16/240,692 · Granted Feb 2, 2021

Methods and systems for generating spatialized audio

Inventors: Samuel C. Mindlin (Brooklyn, NY); Kunal Jathal (Brooklyn, NY); Shan Anis (Jersey City, NJ)
Assignee: Verizon Patent and Licensing Inc.
H04S5/005G08B6/00H04R5/033H04R5/04H04S3/008H04S7/304H04S2400/01H04S2400/11H04S2400/13H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,911,882
App. No.
16/240,692
Granted
Feb 2, 2021
Kind
B2
Abstract

A spatialized audio presentation system selects, based on an orientation of an avatar with respect to a virtual sound source, a head-related impulse response from a library of head-related impulse responses corresponding to different potential orientations of the avatar with respect to the virtual sound source. The spatialized audio presentation system applies the selected head-related impulse response to a sound that is generated by the virtual sound source and is to be presented to a user associated with the avatar. Additionally, the spatialized audio presentation system applies an additional effect to the sound that is to be presented to the user. Corresponding methods and systems are also disclosed.

Claims (95)

1. A method comprising:

selecting, by a spatialized audio presentation system based on an orientation of an avatar with respect to a virtual sound source, a head-related impulse response from a library of head-related impulse responses corresponding to different potential orientations of the avatar with respect to the virtual sound source;

applying, by the spatialized audio presentation system, the selected head-related impulse response to a sound that is generated by the virtual sound source and is to be presented to a user associated with the avatar;

determining, by the spatialized audio presentation system, a source projection parameter indicative of an orientation of a cone of propagation corresponding to the sound and originating at the virtual sound source;

applying, by the spatialized audio presentation system based on the source projection parameter, a source projection effect to the sound, wherein:

the source projection effect is a first effect if the virtual sound source is facing the avatar such that the avatar is located within the cone of propagation, and

the source projection effect is a second effect distinct from the first effect if the virtual sound source is facing away from the avatar such that the avatar is not located within the cone of propagation; and

applying, by the spatialized audio presentation system, an additional effect to the sound that is to be presented to the user.

2. The method of claim 1 , wherein:

the sound that is generated by the virtual sound source and is to be presented to the user is a voice communication spoken by an additional user associated with the virtual sound source and engaged in a virtual experience along with the user;

the virtual sound source that generates the sound to be presented to the user is a virtual avatar of the additional user; and

the cone of propagation originates at a mouth of the virtual avatar of the additional user.

3. The method of claim 1 , further comprising:

determining, by the spatialized audio presentation system, a delay parameter representative of a reflection time of an echo of the sound that is to be presented to the user; and

determining, by the spatialized audio presentation system, a diffusion parameter for the echo based on a material from which the echo virtually reflects when propagating from the virtual sound source to the avatar;

wherein the additional effect applied to the sound is a reverberation effect applied based on the delay parameter and the diffusion parameter.

4. The method of claim 1 , further comprising:

determining, by the spatialized audio presentation system, a first delay parameter representative of a reflection time of an early echo of the sound that is to be presented to the user, the early echo simulating a reflection of the sound from a surface that is a first distance from the avatar; and

determining, by the spatialized audio presentation system, a second delay parameter representative of a reflection time of a late echo of the sound, the late echo simulating a reflection of the sound from a surface that is a second distance from the avatar, the second distance greater than the first distance;

wherein the additional effect applied to the sound is a reverberation effect applied based on the first and second delay parameters.

5. The method of claim 1 , further comprising:

determining, by the spatialized audio presentation system, a first diffusion parameter for a first echo of the sound, the first diffusion parameter determined based on a first material from which the first echo virtually reflects when propagating from the virtual sound source to the avatar; and

determining, by the spatialized audio presentation system, a second diffusion parameter for a second echo of the sound, the second diffusion parameter determined based on a second material that is distinct from the first material and from which the second echo virtually reflects when propagating from the virtual sound source to the avatar;

wherein the additional effect applied to the sound is a reverberation effect applied based on the first and second diffusion parameters.

6. The method of claim 1 , further comprising determining, by the spatialized audio presentation system, an occlusion parameter representative of an effect of a virtual occlusion object on the sound that is to be presented to the user, the virtual occlusion object obstructing a direct sound propagation path between the virtual sound source and the avatar; and

wherein the additional effect applied to the sound is an occlusion effect applied based on the occlusion parameter.

7. The method of claim 1 , further comprising:

identifying, by the spatialized audio presentation system, a distance from the virtual sound source to the avatar; and

determining, by the spatialized audio presentation system based on the distance from the virtual sound source to the avatar, an attenuation parameter representative of an amplitude level fall-off of the sound propagating over the distance from the virtual sound source to the avatar;

wherein the additional effect is applied to the sound by applying, to the sound, the amplitude level fall-off represented by the attenuation parameter.

8. The method of claim 1 , wherein:

the user is associated with the avatar during an augmented reality experience by acting as the avatar; and

the sound virtually propagates from the virtual sound source to the user through an augmented reality space implemented as a physical real-world space surrounding the user during the augmented reality experience.

9. A system comprising:

a memory storing instructions; and

a processor communicatively coupled to the memory and configured to execute the instructions to:

select, based on an orientation of an avatar with respect to a virtual sound source, a head-related impulse response from a library of head-related impulse responses corresponding to different potential orientations of the avatar with respect to the virtual sound source;

apply the selected head-related impulse response to a sound that is generated by the virtual sound source and is to be presented to a user associated with the avatar;

determine a source projection parameter indicative of an orientation of a cone of propagation corresponding to the sound and originating at the virtual sound source;

apply, based on the source projection parameter, a source projection effect to the sound, wherein:

the source projection effect is a first effect if the virtual sound source is facing the avatar such that the avatar is located within the cone of propagation, and

the source projection effect is a second effect distinct from the first effect if the virtual sound source is facing away from the avatar such that the avatar is not located within the cone of propagation; and

apply an additional effect to the sound that is to be presented to the user.

10. The system of claim 9 , wherein:

the sound that is generated by the virtual sound source and is to be presented to the user is a voice communication spoken by an additional user associated with the virtual sound source and engaged in a virtual experience along with the user;

the virtual sound source that generates the sound to be presented to the user is a virtual avatar of the additional user; and

the cone of propagation originates at a mouth of the virtual avatar of the additional user.

11. The system of claim 9 , wherein:

the processor is further configured to execute the instructions to

determine a delay parameter representative of a reflection time of an echo of the sound that is to be presented to the user, and

determine a diffusion parameter for the echo based on a material from which the echo virtually reflects when propagating from the virtual sound source to the avatar; and

the additional effect applied to the sound is a reverberation effect applied based on the delay parameter and the diffusion parameter.

12. The system of claim 9 , wherein:

the processor is further configured to execute the instructions to

determine a first delay parameter representative of a reflection time of an early echo of the sound that is to be presented to the user, the early echo simulating a reflection of the sound from a surface that is a first distance from the avatar, and

determine a second delay parameter representative of a reflection time of a late echo of the sound, the late echo simulating a reflection of the sound from a surface that is a second distance from the avatar, the second distance greater than the first distance; and

the additional effect applied to the sound is a reverberation effect applied based on the first and second delay parameters.

13. The system of claim 9 , wherein:

the processor is further configured to execute the instructions to

determine a first diffusion parameter for a first echo of the sound, the first diffusion parameter determined based on a first material from which the first echo virtually reflects when propagating from the virtual sound source to the avatar, and

determine a second diffusion parameter for a second echo of the sound, the second diffusion parameter determined based on a second material that is distinct from the first material and from which the second echo virtually reflects when propagating from the virtual sound source to the avatar; and

the additional effect applied to the sound is a reverberation effect applied based on the first and second diffusion parameters.

14. The system of claim 9 , wherein:

the processor is further configured to execute the instructions to determine an occlusion parameter representative of an effect of a virtual occlusion object on the sound that is to be presented to the user, the virtual occlusion object obstructing a direct sound propagation path between the virtual sound source and the avatar; and

the additional effect applied to the sound is an occlusion effect applied based on the occlusion parameter.

15. The system of claim 9 , wherein:

the processor is further configured to execute the instructions to

identify a distance from the virtual sound source to the avatar, and

determine, based on the distance from the virtual sound source to the avatar, an attenuation parameter representative of an amplitude level fall-off of the sound propagating over the distance from the virtual sound source to the avatar; and

the additional effect is applied to the sound by applying, to the sound, the amplitude level fall-off represented by the attenuation parameter.

16. The system of claim 9 , wherein:

the user is associated with the avatar during an augmented reality experience by acting as the avatar; and

the sound virtually propagates from the virtual sound source to the user through an augmented reality space implemented as a physical real-world space surrounding the user during the augmented reality experience.

17. A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to:

select, based on an orientation of an avatar with respect to a virtual sound source, a head-related impulse response from a library of head-related impulse responses corresponding to different potential orientations of the avatar with respect to the virtual sound source;

apply the selected head-related impulse response to a sound that is generated by the virtual sound source and is to be presented to a user associated with the avatar;

determine a source projection parameter indicative of an orientation of a cone of propagation corresponding to the sound and originating at the virtual sound source;

apply, based on the source projection parameter, a source projection effect to the sound, wherein:

the source projection effect is a first effect if the virtual sound source is facing the avatar such that the avatar is located within the cone of propagation, and

the source projection effect is a second effect distinct from the first effect if the virtual sound source is facing away from the avatar such that the avatar is not located within the cone of propagation; and

apply an additional effect to the sound that is to be presented to the user.

18. The non-transitory computer-readable medium of claim 17 , wherein:

the sound that is generated by the virtual sound source and is to be presented to the user is a voice communication spoken by an additional user associated with the virtual sound source and engaged in a virtual experience along with the user;

the virtual sound source that generates the sound to be presented to the user is a virtual avatar of the additional user; and

the cone of propagation originates at a mouth of the virtual avatar of the additional user.

19. The non-transitory computer-readable medium of claim 17 , wherein:

the instructions further direct the processor to

determine a delay parameter representative of a reflection time of an echo of the sound that is to be presented to the user, and

determine a diffusion parameter for the echo based on a material from which the echo virtually reflects when propagating from the virtual sound source to the avatar; and

the additional effect applied to the sound is a reverberation effect applied based on the delay parameter and the diffusion parameter.

20. The non-transitory computer-readable medium of claim 17 , wherein:

the instructions further direct the processor to

determine a first delay parameter representative of a reflection time of an early echo of the sound that is to be presented to the user, the early echo simulating a reflection of the sound from a surface that is a first distance from the avatar, and

determine a second delay parameter representative of a reflection time of a late echo of the sound, the late echo simulating a reflection of the sound from a surface that is a second distance from the avatar, the second distance greater than the first distance; and

the additional effect applied to the sound is a reverberation effect applied based on the first and second delay parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 4, 2019
From: MINDLIN, SAMUEL C.; JATHAL, KUNAL; ANIS, SHAN
To: VERIZON PATENT AND LICENSING INC.
Reel/Frame 047910/0124 →
Continuity (2)
Continuation 15856359 · Dec 28, 2017
Related Publication 20190208345A1 · Jul 4, 2019