Object-oriented audio streaming system
View Patent ↗Systems and methods for providing object-oriented audio are described. Audio objects can be created by associating sound sources with attributes of those sound sources, such as location, velocity, directivity, and the like. Audio objects can be used in place of or in addition to channels to distribute sound, for example, by streaming the audio objects over a network to a client device. The objects can define their locations in space with associated two or three dimensional coordinates. The objects can be adaptively streamed to the client device based on available network or client device resources. A renderer on the client device can use the attributes of the objects to determine how to render the objects. The renderer can further adapt the playback of the objects based on information about a rendering environment of the client device. Various examples of audio object creation techniques are also described.
1. A method of adapting transmission of an object-oriented audio stream, the method comprising:
receiving a request from a remote computer system for audio content;
receiving resource information from the remote computer system regarding available computing resources of the remote computer system;
programmatically selecting, with one or more processors, a plurality of audio objects associated with the audio content for transmission to the remote computer system based at least in part on the resource information, said selecting comprising selecting relatively more of the audio objects for rendering when the remote computer system has relatively more available computing resources and selecting relatively fewer of the audio objects for rendering when the remote computer system has relatively fewer available computing resources;
the plurality of audio objects comprising:
channel objects, each channel object comprising a channel of audio, and
dynamic objects, each dynamic object comprising metadata, the metadata for each of the dynamic objects comprising object attributes, the object attributes for each of the dynamic object comprising information regarding one or more of the following: location of the object, velocity of the object, and occlusion of the object; and
providing the channel objects and the dynamic objects to the remote computer system, thereby facilitating backwards compatibility with the remote computer system if the remote computer system is a fixed channel system enabling the the remote computer system to achieve enhanced rendering if the remote computer system is capable of also rendering the dynamic objects.
2. The method of claim 1 , wherein at least one of the dynamic objects comprises psychoacoustic information configured to enhance at least one of the channel objects.
3. The method of claim 1 , wherein at least one of the dynamic objects comprises a background object configured to enhance an audio scene.
4. The method of claim 1 , wherein at least one of the audio objects comprises dialog.
5. The method of claim 1 , wherein the channel objects do not comprise metadata that represents object attributes.
6. The method of claim 1 , wherein said selecting comprises selecting the channel objects to be provided continuously during said providing while selecting the dynamic objects to be provided whenever the dynamic objects are available.
7. A system for adapting transmission of an object-oriented audio stream, the system comprising:
a resource monitor configured to receive resource information from a remote computer system regarding available computing resources of the remote computer system;
an object-oriented encoder comprising one or more processors configured to select a plurality of audio objects for transmission to the remote computer system by selecting relatively more of the audio objects for rendering when the remote computer system has relatively more available computing resources and selecting relatively fewer of the audio objects for rendering when the remote computer system has relatively fewer available computing resources,
the plurality of audio objects comprising:
channel objects, each channel object comprising a channel of audio, and
dynamic objects, each dynamic object comprising metadata, the metadata for each of the dynamic objects comprising object attributes, the object attributes for each of the dynamic object comprising information regarding one or more of the following: location of the object, velocity of the object, and occlusion of the object; and
a streaming module configured to provide the channel objects and the dynamic objects to the remote computer system, thereby facilitating backwards compatibility with the remote computer system if the remote computer system is a fixed channel system and enabling the remote computer system to achieve enhanced rendering if the remote computer system is capable of also rendering the dynamic objects.
8. The system of claim 7 , wherein at least one of the dynamic objects comprises psychoacoustic information configured to enhance at least one of the channel objects.
9. The system of claim 7 , wherein at least one of the dynamic objects comprises a background object configured to enhance an audio scene.
10. The system of claim 7 , wherein at least one of the audio objects comprises dialog.
11. The system of claim 7 , wherein the channel objects do not comprise metadata that represents object attributes.
12. The system of claim 7 , wherein the object-oriented encoder is further configured to select the plurality of audio objects by at least selecting the channel objects to be provided continuously during said providing while selecting the dynamic objects to be provided whenever the dynamic objects are available.