Frame coding for spatial audio data
The techniques disclosed herein provide apparatuses and related methods for the communication of spatial audio and related metadata. In some implementations, a source provides prerecorded spatial audio that has embedded metadata. A computing device processes the prerecorded spatial audio to generate an audio codec that is segmented to include a first section of audio data and a second section that includes metadata extracted from the prerecorded spatial audio. The generated audio codec may be received by a device that includes an encoder. The encoder may process the generated audio codec to generate audio data that includes the metadata.
1. A computing device, comprising:
a processor;
a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to:
receive a spatial audio stream;
generate audio data from the spatial audio stream by removing at least one associated metadata component from a portion of the spatial audio stream, the at least one associated metadata component comprising positional metadata used to render at least a portion of the audio data in a three-dimensional space;
store the at least one associated metadata component in a storage associated with the computing device; and
generate a codec frame having a predetermined length and comprising first and second separated sections, the first section including at least a portion of the audio data and the second section including the at least one associated metadata component removed from the spatial audio stream.
2. The computing device according to claim 1 , wherein the spatial audio stream includes the audio data and a plurality of associated metadata components, the processor to extract the plurality of associated metadata components, store the plurality of associated metadata components, and generate the codec frame including the plurality of associated metadata components disposed in the second section of the codec frame.
3. The computing device according to claim 2 , wherein the plurality of associated metadata components comprises the positional metadata including one or more coordinates to render the at least a portion of the audio data in the three-dimensional space, a gain of the at least a portion of audio data, and calibration information for one or more audio rendering elements to playback the at least a portion of the audio data.
4. The computing device according to claim 1 , wherein the audio data is pulse code modulation (PCM) audio data and the predetermined length is 32 ms and comprises 1536 PCM samples.
5. The computing device according to claim 1 , wherein the computer-executable instructions, when executed by the processor, cause the processor to advertise a metadata format identification indicating that the computing device is to generate the codec frame having the predetermined length and comprising the first and second separated sections.
6. The computing device according to claim 5 , wherein the computer-executable instructions, when executed by the processor, cause the computing device to receive an acknowledgment that an encoder associated with an endpoint device supports the codec frame having the predetermined length and comprising the first and second separated sections.
7. The computing device according to claim 6 , wherein the acknowledgment is received in response to the metadata format identification advertised by the computing device.
8. The computing device according to claim 1 , wherein the spatial audio stream is associated with prerecorded media provided by a streaming service provider that provides streaming media content to endpoint devices and users of the endpoint devices.
9. A computer-implemented method, comprising:
receiving a spatial audio stream;
generating audio data from the spatial audio stream by removing at least one associated metadata component from a portion of the spatial audio stream, the at least one associated metadata component comprising positional metadata used to render at least a portion of the audio data in a three-dimensional space;
storing the at least one associated metadata component in a storage associated with the computing device; and
generating a codec frame having a predetermined length and comprising first and second separated sections, the first section including at least a portion of the audio data and the second section including the at least one associated metadata component removed from the spatial audio stream.
10. The computer-implemented method of claim 9 , wherein the spatial audio stream includes the audio data and a plurality of associated metadata components, the processor to extract the plurality of associated metadata components, store the plurality of associated metadata components, and generate the codec frame including the plurality of associated metadata components disposed in the second section of the codec frame.
11. The computer-implemented method of claim 10 , wherein the plurality of associated metadata components comprises the positional metadata including one or more coordinates to render the at least a portion of the audio data in the three-dimensional space, a gain of the at least a portion of audio data, and calibration information for one or more audio rendering elements to playback the at least a portion of the audio data.
12. The computer-implemented method of claim 9 , wherein the audio data is pulse code modulation (PCM) audio data and the predetermined length is 32 ms and comprises 1536 PCM samples.
13. The computer-implemented method of claim 9 , further comprising advertising a metadata format identification indicating that the computing device is to generate the codec frame having the predetermined length and comprising the first and second separated sections.
14. The computer-implemented method of claim 13 , further comprising receiving an acknowledgment that an encoder associated with an endpoint device supports the codec frame having the predetermined length and comprising the first and second separated sections.
15. The computer-implemented method of claim 14 , wherein the acknowledgment is received in response to the metadata format identification advertised by the computing device.
16. A computer-readable storage medium in communication with a processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to:
receive a spatial audio stream;
generate audio data from the spatial audio stream by removing at least one associated metadata component from a portion of the spatial audio stream, the at least one associated metadata component comprising positional metadata used to render at least a portion of the audio data in a three-dimensional space;
store the at least one associated metadata component in a storage associated with the computing device; and
generate a codec frame having a predetermined length and comprising first and second separated sections, the first section including at least a portion of the audio data and the second section including the at least one associated metadata component removed from the spatial audio stream.
17. The computer-readable storage medium of claim 16 , wherein the spatial audio stream includes the audio data and a plurality of associated metadata components, the processor to extract the plurality of associated metadata components, store the plurality of associated metadata components, and generate the codec frame including the plurality of associated metadata components disposed in the second section of the codec frame.
18. The computer-readable storage medium of claim 17 , wherein the plurality of associated metadata components comprises the positional metadata including one or more coordinates to render the at least a portion of the audio data in the three-dimensional space, a gain of the at least a portion of audio data, and calibration information for one or more audio rendering elements to playback the at least a portion of the audio data.
19. The computer-readable storage medium of claim 16 , wherein the audio data is pulse code modulation (PCM) audio data and the predetermined length is 32 ms and comprises 1536 PCM samples.
20. The computer-readable storage medium of claim 16 , wherein the computer-executable instructions, when executed by the processor, cause the processor to advertise a metadata format identification indicating that the computing device is to generate the codec frame having the predetermined length and comprising the first and second separated sections.