System for adaptively streaming audio objects
View Patent ↗Systems and methods for providing object-oriented audio are described. Audio objects can be created by associating sound sources with attributes of those sound sources, such as location, velocity, directivity, and the like. Audio objects can be used in place of or in addition to channels to distribute sound, for example, by streaming the audio objects over a network to a client device. The objects can define their locations in space with associated two or three dimensional coordinates. The objects can be adaptively streamed to the client device based on available network or client device resources. A renderer on the client device can use the attributes of the objects to determine how to render the objects. The renderer can further adapt the playback of the objects based on information about a rendering environment of the client device. Various examples of audio object creation techniques are also described.
1. A system for adapting transmission and rendering of an object-oriented audio stream, the system comprising:
a computing resource monitor implemented by a computer system, the computing resource monitor configured to access resource information regarding available computing resources of the computer system; and
a rendering module implemented by the computer system, the rendering module configured to:
receive an audio stream from a remote server, the audio stream comprising a plurality of audio objects, the plurality of audio objects comprising:
channel objects, each channel object comprising a channel of audio, and
dynamic objects, each dynamic object comprising metadata, the metadata for each of the dynamic objects comprising object attributes and a priority value;
access environment information regarding the available computing resources of the computer system;
select with the computer system, one or more of the plurality of audio objects based at least partly on the available computing resources and the priority values of the dynamic objects, said selecting comprising selecting relatively more of the dynamic objects for rendering when the available computing resources have relatively more audio-rendering capability and selecting relatively fewer of the dynamic objects for rendering when the available computing resources have relatively lower audio-rendering capability;
render the selected one or more audio objects to produce output audio;
supply the output audio to one or more speakers; and
communicate the information regarding the computing resources of the computer system to the remote server to thereby enable the remote server to adjust an amount of the objects that are transmitted by the remote server to the computer system.
2. A method of adapting rendering of an object-oriented audio stream, the method comprising:
receiving, with a computer system, an audio stream from a remote server over a network, the audio stream comprising a plurality of audio objects, the plurality of audio objects comprising:
channel objects, each channel object comprising a channel of audio, and
dynamic objects, each dynamic object comprising metadata, the metadata for each of the dynamic objects comprising object attributes and a priority value, the object attributes for each of the dynamic object comprising information regarding one or more of the following: location of the object, velocity of the object, and occlusion of the object;
accessing environment information regarding a playback environment associated with the computer system, the information regarding the playback environment comprising information regarding audio-rendering capabilities of the computer system;
selecting, with the computer system, one or more of the dynamic objects based at least partly on the environment information and the priority values of the dynamic objects, said selecting comprising selecting relatively more of the dynamic objects for rendering when the playback environment has relatively more audio-rendering capability and selecting relatively fewer of the dynamic objects for rendering when the playback environment has relatively lower audio-rendering capability;
communicating the information regarding the computing resources of the computer system to the remote server to thereby enable the remote server to adjust an amount of the dynamic objects that are transmitted by the remote server to the computer system;
rendering the selected one or more dynamic objects and the channel objects to produce output audio; and
supplying the output audio to one or more speakers.
3. The method of claim 2 , wherein the environment information comprises information regarding one or more of the following: a number of speakers connected to the computer system, a type of device of the computer system, psychoacoustic processing capability of the computing system, software installed on the computing system, and hardware characteristics of the computing system.
4. The method of claim 2 , wherein said rendering comprises applying a psychoacoustic enhancement to the one or more audio objects in response to determining a number of speakers in the playback environment.