Methods, apparatus and systems for a pre-rendered signal for audio rendering
The present disclosure relates to a method of decoding audio scene content from a bitstream by a decoder that includes an audio renderer with one or more rendering tools. The method comprises receiving the bitstream, decoding a description of an audio scene from the bitstream, determining one or more effective audio elements from the description of the audio scene, determining effective audio element information indicative of effective audio element positions of the one or more effective audio elements from the description of the audio scene, decoding a rendering mode indication from the bitstream, wherein the rendering mode indication is indicative of whether the one or more effective audio elements represent a sound field obtained from pre-rendered audio elements and should be rendered using a predetermined rendering mode, and in response to the rendering mode indication indicating that the one or more effective audio elements represent the sound field obtained from pre-rendered audio elements and should be rendered using the predetermined rendering mode, rendering the one or more effective audio elements using the predetermined rendering mode, wherein rendering the one or more effective audio elements using the predetermined rendering mode takes into account the effective audio element information, and wherein the predetermined rendering mode defines a predetermined configuration of the rendering tools for controlling an impact of an acoustic environment of the audio scene on the rendering output. The disclosure further relates to a method of generating audio scene content and a method of encoding audio scene content into a bitstream.
1 . A method of decoding audio scene content from a bitstream by a decoder that includes an audio renderer with one or more rendering tools, the method comprising:
receiving the bitstream from an encoder, wherein the bitstream includes one or more effective audio elements, effective audio element information, a rendering mode indication, and listener position area information,
wherein the one or more effective audio elements encapsulate an impact of an acoustic environment including two or more of reverberation, echo, direct reflection, or acoustic occlusion, wherein each effective audio element is a virtual audio object, wherein the effective audio element information is indicative of effective audio element positions of the one or more effective audio elements, wherein the effective audio element positions are different from positions of original audio objects,
wherein the rendering mode indication is indicative of whether the one or more effective audio elements represent a sound field obtained from pre-rendered audio elements and should be rendered using a predetermined rendering mode, wherein the listener position area information is indicative of a listener position area in the acoustic environment; and
in response to the rendering mode indication indicating that the one or more effective audio elements represent the sound field obtained from the pre-rendered audio elements and should be rendered using the predetermined rendering mode, rendering the one or more effective audio elements using the predetermined rendering mode within the listener position area,
wherein rendering the one or more effective audio elements using the predetermined rendering mode takes into account the effective audio element information, and wherein the predetermined rendering mode defines a predetermined configuration of the rendering tools for controlling the impact of the acoustic environment on a rendering output.
2 . The method according to claim 1 , wherein rendering the one or more effective audio elements using the predetermined rendering mode comprises applying sound attenuation modeling in accordance with respective distances between a listener position and an effective audio element position of each effective audio element.
3 . The method according to claim 1 , wherein the predetermined rendering mode depends on the listener position area.
4 . The method according to claim 1 , wherein the acoustic environment is a virtual reality (VR), augmented reality (AR), or mixed reality (MR) acoustic environment.
5 . A non-transitory computer-readable storage medium including instructions for causing a processor that carries out the instructions to perform the method according to claim 1 .
6 . An apparatus for audio decoding, the apparatus comprising:
an audio decoder for receiving a bitstream from an encoder, wherein the bitstream includes one or more effective audio elements, effective audio element information, a rendering mode indication, and listener position area information,
wherein the one or more effective audio elements encapsulate an impact of an acoustic environment including two or more of reverberation, echo, direct reflection, or acoustic occlusion, wherein each effective audio element is a virtual audio object, wherein the effective audio element information is indicative of effective audio element positions of the one or more effective audio elements, wherein the effective audio element positions are different from positions of original audio objects,
wherein the rendering mode indication is indicative of whether the one or more effective audio elements represent a sound field obtained from pre-rendered audio elements and should be rendered using a predetermined rendering mode, wherein the listener position area information is indicative of a listener position area in the acoustic environment; and
a renderer for, in response to the rendering mode indication indicating that the one or more effective audio elements represent the sound field obtained from the pre-rendered audio elements and should be rendered using the predetermined rendering mode, rendering the one or more effective audio elements using the predetermined rendering mode within the listener position area,
wherein rendering the one or more effective audio elements using the predetermined rendering mode takes into account the effective audio element information, and wherein the predetermined rendering mode defines a predetermined configuration of one or more rendering tools for controlling the impact of the acoustic environment on a rendering output.
7 . The method according to claim 1 , wherein rendering the one or more effective audio elements using the predetermined rendering mode comprises adding artificial acoustic effects to the one or more effective audio elements.
8 . The method according to claim 7 , wherein the artificial acoustic effects comprise one or more of the reverberation, the echo, the direct reflection, or the acoustic occlusion.
9 . The method according to claim 1 , wherein the rendering mode indication comprises one or more control parameters for configuring the one or more rendering tools.
10 . The method according to claim 1 , wherein the effective audio element information comprises information indicative of respective sound radiation patterns of the effective audio elements,
the method further comprising:
determining a gain for each of the effective audio elements based on respective sound radiation patterns; and
applying a respective gain to each of the effective audio elements.
11 . The method according to claim 10 , wherein determining the gain for each of the effective audio elements comprises determining the gain based on an angle between a distance vector from each effective audio element to a listener position within the listener position area and a radiation direction vector indicating a radiation direction of each effective audio element.
12 . The method according to claim 1 , wherein the rendering mode indication indicates a respective predetermined rendering mode for each of the effective audio elements.