IP Library › Granted Patent US 11,405,741
Granted Patent B2
US 11,405,741 · App. 16/768,671 · Granted Aug 2, 2022

Method and system for handling global transitions between listening positions in a virtual reality environment

Inventors: Leon Terentiv (Erlangen, DE); Christof Fersch (Neumarkt, DE); Daniel Fischer (Fuerth, DE)
Assignee: Dolby International AB
H04S7/303G06T19/006H04S7/307
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,405,741
App. No.
16/768,671
Granted
Aug 2, 2022
Kind
B2
Abstract

A method ( 900 ) for rendering audio in a virtual reality rendering environment ( 180 ) is described. The method ( 900 ) comprises rendering ( 901 ) an origin audio signal of an origin audio source ( 113 ) of an origin audio scene ( 111 ) from an origin source position on a sphere ( 114 ) around a listening position ( 201 ) of a listener ( 181 ). Furthermore, the method ( 900 ) comprises determining ( 902 ) that the listener ( 181 ) moves from the listening position ( 201 ) within the origin audio scene ( 111 ) to a listening position ( 202 ) within a different destination audio scene ( 112 ). In addition, the method ( 900 ) comprises applying ( 903 ) a fade-out gain to the origin audio signal to determine a modified origin audio signal, and rendering ( 903 ) the modified origin audio signal of the origin audio source ( 113 ) from the origin source position on the sphere ( 114 ) around the listening position ( 201, 202 ).

Claims (62)

1. A method for rendering audio in a virtual reality rendering environment using a 3DoF renderer, the method comprising,

rendering, by the 3DoF renderer, an origin audio signal of an origin audio source of an origin audio scene from an origin source position on a sphere around an origin listening position of a listener within a virtual reality rendering environment;

determining that the listener moves within the virtual reality rendering environment from the origin listening position within the origin audio scene to a destination listening position within a different destination audio scene;

responsive to the determination that the listener moves from the origin listening position within the origin audio scene to the destination listening position within the destination audio scene, applying a fade-out gain to the origin audio signal to determine a modified origin audio signal; and

rendering, by the 3DoF renderer, the modified origin audio signal of the origin audio source from the origin source position on the sphere around the origin listening position; wherein the method further comprises:

determining a destination audio signal of a destination audio source of the destination audio scene;

determining a destination source position on the sphere around the destination listening position;

applying a fade-in gain to the destination audio signal to determine a modified destination audio signal; and

rendering, by the 3DoF renderer, the modified destination audio signal of the destination audio source from the destination source position on the sphere around the destination listening position.

2. The method of claim 1 , wherein the modified origin audio signal is rendered from the same position relative to the listener throughout the movement from the origin listening position within the origin audio scene to the destination listening position within the destination audio scene.

3. The method of claim 1 , wherein the destination audio scene does not include the origin audio source.

4. The method of claim 1 , wherein the method comprises,

determining that the listener moves from the origin audio scene to the destination audio scene during a transition time interval;

determining an intermediate time instant within the transition time interval; and

determining the fade-out gain based on a relative location of the intermediate time instant within the transition time interval.

5. The method of claim 4 , wherein

a respective fade-out gain is determined for each of a plurality of intermediate time instants within the transition time interval based on the relative location of the respective intermediate time instant within the transition interval; and

the respective determined fade-out gain is applied to the origin audio signal to determine a respective modified origin audio signal for each of the plurality of intermediate time constants within the transition time interval.

6. The method of claim 1 , wherein the fade-out gain is different from zero up until the destination listening position within the destination audio scene is reached.

7. The method of claim 4 , wherein

the method comprises providing a fade-out function which indicates the fade-out gain at different intermediate time instants within the transition time interval; and

the fade-out function is such that the fade-out gain decreases with progressing intermediate time instants.

8. The method of claim 7 , wherein the fade-out function is such that

the origin audio signal remains unmodified at a beginning of the transition time interval; and/or

the origin audio signal is increasingly attenuated at progressing intermediate time instants; and/or

the origin audio signal is fully attenuated at an end of the transition time interval.

9. The method of claim 1 , wherein the method comprises

maintaining the origin source position of the origin audio source on the sphere around the origin listening position as the listener moves within the virtual reality rendering environment from the origin audio scene to the destination audio scene; and/or

maintaining the origin listening position unchanged as the listener moves within the virtual reality rendering environment from the origin audio scene to the destination audio scene.

10. The method of claim 1 , wherein the modified destination audio signal is rendered from the same position relative to the listener throughout the movement from the origin listening position within the origin audio scene to the destination listening position within the destination audio scene.

11. The method of claim 10 , wherein the origin audio scene does not include the destination audio source.

12. The method of claim 1 , wherein the method comprises,

determining that the listener moves from the origin audio scene to the destination audio scene during a transition time interval;

determining an intermediate time instant within the transition time interval; and

determining the fade-in gain based on a relative location of the intermediate time instant within the transition time interval.

13. The method of claim 12 , wherein:

a respective fade-in gain is determined for each of a plurality of intermediate time instants within the transition time interval based on the relative location of the respective intermediate time instant within the transition interval; and

the respective determined fade-in gain is applied to the destination audio signal to determine a respective modified destination audio signal for each of the plurality of intermediate time constants within the transition time interval.

14. The method of claim 1 , wherein the fade-in gain is different from zero during the movement from the origin audio scene to the destination audio scene.

15. The method of claim 12 , wherein

the method comprises providing a fade-in function which indicates the fade-in gain at different intermediate time instants within the transition time interval; and

the fade-in function is such that the fade-in gain increases with progressing intermediate time instants.

16. The method of claim 15 , wherein the fade-in function is such that

the destination audio signal remains unmodified at an end of the transition time interval; and/or

the destination audio signal is decreasingly attenuated at progressing intermediate time instants; and/or

the destination audio signal is fully attenuated at a beginning of the transition time interval.

17. The method of claim 1 , wherein the method comprises

maintaining the destination source position of the destination audio source on the sphere around the destination listening position as the listener moves within the virtual reality rendering environment from the origin audio scene to the destination audio scene; and/or

maintaining the destination listening position unchanged as the listener moves within the virtual reality rendering environment from the origin audio scene to the destination audio scene.

18. The method of claim 7 , wherein

the method comprises providing a fade-in function which indicates the fade-in gain at different intermediate time instants within the transition time interval;

the fade-in function is such that the fade-in gain increases with progressing intermediate time instants; and

the fade-out function and the fade-in function in combination provide a constant gain for a plurality of different intermediate time instants.

19. The method of claim 7 , wherein

the method comprises providing a fade-in function which indicates the fade-in gain at different intermediate time instants within the transition time interval;

the fade-in function is such that the fade-in gain increases with progressing intermediate time instants; and

the fade-out function and/or the fade-in function:

are derived from a bitstream which is indicative of the origin audio signal and/or the destination audio signal; and/or

are derived from a storage unit of a virtual reality audio render configured to render the origin audio signal and/or the destination audio signal within the virtual reality rendering environment.

20. The method of claim 1 , wherein the method comprises receiving an indication that the listener moves from the origin audio scene to the destination audio scene.

21. The method of claim 20 , wherein the indication comprises a flag.

22. The method of claim 1 , wherein the method comprises sending an indication that the listener moves from the origin audio scene to the destination audio scene to an encoder; wherein the encoder is configured to generate a bitstream which is indicative of the origin audio signal.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 6, 2020
From: TERENTIV, LEON; FERSCH, CHRISTOF; FISCHER, DANIEL
To: DOLBY INTERNATIONAL AB
Reel/Frame 053987/0564 →
Priority Claims (1)
EP 17208088 · Dec 18, 2017 · regional
Continuity (2)
Provisional Application 62599841 · Dec 18, 2017
Related Publication 20210006924A1 · Jan 7, 2021