System and method for adaptive audio signal generation, coding and rendering
Embodiments are described for an adaptive audio system that processes audio data comprising a number of independent monophonic audio streams. One or more of the streams has associated with it metadata that specifies whether the stream is a channel-based or object-based stream. Channel-based streams have rendering information encoded by means of channel name; and the object-based streams have location information encoded through location expressions encoded in the associated metadata. A codec packages the independent audio streams into a single serial bitstream that contains all of the audio data. This configuration allows for the sound to be rendered according to an allocentric frame of reference, in which the rendering location of a sound is based on the characteristics of the playback environment (e.g., room size, shape, etc.) to correspond to the mixer's intent. The object position metadata contains the appropriate allocentric frame of reference information required to play the sound correctly using the available speaker positions in a room that is set up to play the adaptive audio content.
1. A system for processing audio signals, comprising a rendering system configured to:
receive a bitstream comprising encoded audio data representing a plurality of monophonic audio streams, and further comprising metadata associated with each of the monophonic audio streams and indicating a playback location of a respective monophonic audio stream, wherein at least some of the plurality of monophonic audio streams are identified as object-based audio, and wherein the playback location of an object-based monophonic audio stream comprises a location in a three-dimensional space, wherein at least some other of the plurality of monophonic audio streams are identified as channel-based audio, and wherein the playback location of a channel-based monophonic audio stream comprises a location in the three-dimensional space;
decode the encoded audio data to provide the plurality of monophonic audio streams; and
render the plurality of monophonic audio streams to a plurality of speaker feeds corresponding to speakers in a playback environment, wherein the speakers are placed at specific positions within the playback environment, and wherein one or more additional metadata elements associated with each respective object-based monophonic audio stream indicate whether rendering the respective monophonic audio stream into one or more specific speaker feeds of the plurality of speaker feeds is prohibited, such that the respective object-based monophonic audio stream is not rendered into any of the one or more specific speaker feeds of the plurality of speaker feeds.
2. The system of claim 1 , wherein the metadata elements associated with each object-based monophonic audio stream further indicate spatial parameters controlling the playback of a corresponding sound component comprising one or more of: sound position, sound width, and sound velocity.
3. The system of claim 1 , wherein the playback location for each of the plurality of object-based monophonic audio streams is independently specified with respect to either an egocentric frame of reference or an allocentric frame of reference, wherein the egocentric frame of reference is taken in relation to a listener in the playback environment, and wherein the allocentric frame of reference is taken with respect to a characteristic of the playback environment.
4. A method for authoring audio content for rendering, comprising:
receiving a plurality of audio signals;
generating a plurality of monophonic audio streams and metadata associated with each of the audio streams and indicating a playback location of a respective monophonic audio stream, wherein at least some of the plurality of monophonic audio streams are identified as object-based audio, and wherein the playback location of the object-based audio comprises a location in a three-dimensional space, wherein at least some other of the plurality of monophonic audio streams are identified as channel-based audio, and wherein the playback location of a channel-based monophonic audio stream comprises a location in the three-dimensional space;
encoding the plurality of monophonic audio streams to provide encoded audio data; and
encapsulating the encoded audio data and the metadata in a bitstream for transmission to a rendering system configured to render the plurality of monophonic audio streams to a plurality of speaker feeds corresponding to speakers in a playback environment, wherein the speakers are placed at specific positions within the playback environment, and wherein one or more additional metadata elements associated with each respective object-based monophonic audio stream indicate whether rendering the respective monophonic audio stream into one or more specific speaker feeds of the plurality of speaker feeds is prohibited, such that the respective object-based monophonic audio stream is not rendered into any of the one or more specific speaker feeds of the plurality of speaker feeds.
5. A system for authoring audio content for rendering configured to perform the method of claim 4 .
6. A non-transitory computer readable storage medium comprising a sequence of instructions, wherein, when executed by a system for processing audio signals, the sequence of instructions causes the system to perform the method of claim 4 .
7. A method for rendering audio signals, comprising:
receiving a bitstream comprising encoded audio data representing a plurality of monophonic audio streams, and further comprising metadata associated with each of the audio streams and indicating a playback location of a respective monophonic audio stream, wherein at least some of the plurality of monophonic audio streams are identified as object-based audio, and wherein the playback location of an object-based monophonic audio stream comprises a location in three-dimensional space, wherein at least some other of the plurality of monophonic audio streams are identified as channel-based audio, and wherein the playback location of a channel-based monophonic audio stream comprises a location in the three-dimensional space;
decoding the encoded audio data to provide the plurality of monophonic audio bitstreams; and
rendering the plurality of monophonic audio streams to a plurality of speaker feeds corresponding to speakers in a playback environment, wherein the speakers are placed at specific positions within the playback environment, and wherein one or more additional metadata elements associated with each respective object-based monophonic audio stream indicate whether rendering the respective monophonic audio stream into one or more specific speaker feeds of the plurality of speaker feeds is prohibited, such that the respective object-based monophonic audio stream is not rendered into any of the one or more specific speaker feeds of the plurality of speaker feeds.
8. The method of claim 7 , wherein the metadata elements associated with each object-based monophonic audio stream further indicate spatial parameters controlling the playback of a corresponding sound component comprising one or more of: sound position, sound width, and sound velocity.
9. The method of claim 7 , wherein the playback location for each of the plurality of object-based monophonic audio streams comprises a spatial position relative to a screen within a playback environment, or a surface that encloses the playback environment, and wherein the surface comprises a front plane, a back plane, a left plane, right plane, an upper plane, and a lower plane, and/or is independently specified with respect to either an egocentric frame of reference or an allocentric frame of reference, wherein the egocentric frame of reference is taken in relation to a listener in the playback environment, and wherein the allocentric frame of reference is taken with respect to a characteristic of the playback environment.
10. A non-transitory computer readable storage medium comprising a sequence of instructions, wherein, when executed by a system for processing audio signals, the sequence of instructions causes the system to perform the method of claim 7 .