IP Library › Granted Patent US 11,381,886
Granted Patent B2
US 11,381,886 · App. 15/931,422 · Granted Jul 5, 2022

Data processor and transport of user control data to audio decoders and renderers

Inventors: Stephan Schreiner (Birgland, DE); Simone Neukam (Kalchreuth, DE); Harald Fuchs (Roettenbach, DE); Jan Plogsties (Fuerth, DE); Stefan Doehla (Erlangen, DE)
Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
H04N21/8106G10L19/00G10L19/167H04N21/435H04N21/4363H04N21/4394H04N21/44222H04N21/44227H04N21/4852
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,381,886
App. No.
15/931,422
Granted
Jul 5, 2022
Kind
B2
Abstract

Audio data processor, having: a receiver interface for receiving encoded audio data and metadata related to the encoded audio data; a metadata parser for parsing the metadata to determine an audio data manipulation possibility; an interaction interface for receiving an interaction input and for generating, from the interaction input, interaction control data related to the audio data manipulation possibility; and a data stream generator for obtaining the interaction control data and the encoded audio data and the metadata and for generating an output data stream, the output data stream having the encoded audio data, at least a portion of the metadata, and the interaction control data.

Claims (46)

1. Audio data processor for processing packetized audio data, the audio data processor comprising:

a receiver interface for receiving encoded audio data comprising audio objects and metadata related to the audio objects;

a metadata parser for parsing the metadata to determine an audio data manipulation possibility of the audio objects;

an interaction interface for receiving an interaction input and for generating, from the interaction input, interaction control data related to the audio data manipulation possibility for manipulating the audio objects externally from a decoder; and

a data stream generator for acquiring the interaction control data and the encoded audio data and the metadata and for generating an output data stream by adding an additional data portion comprising the interaction control data to the output data stream, wherein the original metadata and the encoded audio data are not modified, such that the output data stream being again a valid encoded audio stream comprising the still encoded audio data, the metadata, and the added interaction control data,

wherein the output data stream is sent to the decoder, wherein the presence of the interaction control data in the output data stream enables said decoder to identify that an interaction has happened.

2. The audio data processor of claim 1 , wherein the additional data portion is comprised in a packet type being different from packet types for the encoded audio data and the metadata.

3. The audio data processor of claim 1 , wherein the interaction interface provides two interaction modes,

wherein a first interaction mode is an advanced interaction mode that is signaled for each element group being present in an audio scene for enabling the user to freely choose which groups to play back and to interact with all of them, and

wherein a second mode is basic interaction mode comprising Group Presets, for enabling the user to choose one of the Group Presets as a preset.

4. Audio data processor of claim 1 , wherein the encoded audio data comprises separate encoded audio objects, wherein at least a portion of the metadata is related to a corresponding audio object,

wherein the metadata parser is configured to parse the corresponding portion for the encoded audio objects to determine, for at least an audio object, the object manipulation possibility,

wherein the interaction interface is configured to generate, for the at least one encoded audio object, the interaction control data from the interaction input related to the at least one encoded audio object.

5. Audio data processor of claim 1 , wherein the interaction interface is configured to present, to a user, the audio data manipulation possibility derived from the metadata by the metadata parser, and to receive, from the user, a user input on the specific data manipulation of the data manipulation possibility.

6. Audio data processor of claim 1 ,

wherein the data stream generator is configured to generate, in the output data stream, the interaction control data in the same format as the metadata.

7. Audio data processor of claim 1 ,

wherein the data stream generator is configured to associate, with the interaction control data, an identifier in the output data stream, the identifier being different from an identifier associated with the metadata.

8. Audio data processor of claim 1 ,

wherein the data stream generator is configured to add, to the interaction control data, signature data, the signature data indicating information on an application, a device or a user performing an audio data manipulation or providing the interaction input.

9. Audio data processor of claim 1 ,

wherein the metadata parser is configured to identify a disabling possibility for one or more audio objects represented by the encoded audio data,

wherein the interaction interface is configured for receiving a disabling information for the one or more audio objects, and

wherein the data stream generator is configured for marking the one or more audio objects as disabled in the interaction control data or for removing the disabled one or more audio objects from the encoded audio data so that the output data stream does not comprise encoded audio data for the disabled one or more audio objects.

10. Audio data processor of claim 1 , wherein the data stream generator is configured to dynamically generate the output data stream, wherein in response to a new interaction input, the interaction control data is updated to match the new interaction input, and wherein the data stream generator is configured to comprise the updated interaction control data in the output data stream.

11. Audio data processor of claim 1 , wherein the receiver interface is configured to receive a main audio data stream comprising the encoded audio data and metadata related to the encoded audio data, and to additionally receive optional audio data comprising an optional audio object,

wherein the metadata related to said optional audio object is comprised in said main audio data stream.

12. Audio data processor of claim 1 ,

wherein the metadata parser is configured to determine the audio manipulation possibility for an optional audio object not comprised in the encoded audio data,

wherein the interaction interface is configured to receive an interaction input for the optional audio object, and

wherein the receiver interface is configured to request audio data for the optional audio object from an audio data provider or to receive the audio data for the optional audio object from a different substream comprised in a broadcast stream or an internet protocol connection.

13. Audio data processor of claim 1 ,

wherein the data stream generator is configured to add, into the output data stream, fill data in a fill data packet type, wherein an amount of fill data is determined based on a data rate requirement determined by an output interface of the audio data processor.

14. Audio data processor of claim 1 being implemented as a separate first device that is separated from a second device which is configured to receive the processed, but still encoded, audio data from the first device for decoding said audio data, wherein the receiver interface forms an input to the separate first device via a wired or wireless connection, wherein the audio data processor further comprises an output interface connected to the data stream generator, the output interface being configured for outputting the output data stream, wherein the output interface performs an output of the separate first device and comprises a wireless interface or a wire connector.

15. Method for processing packetized audio data, the method comprising:

receiving encoded audio data comprising audio objects and metadata related to the audio objects;

parsing the metadata to determine an audio data manipulation possibility of the audio objects;

receiving an interaction input and generating, from the interaction input, interaction control data related to the audio data manipulation possibility for manipulating the audio objects externally from a decoder; and

acquiring the interaction control data and the encoded audio data and the metadata and generating an output data stream by adding an additional data portion comprising the interaction control data to the output data stream, wherein the original metadata and the encoded audio data are not modified, such that the output data stream being again a valid encoded audio stream comprising the still encoded audio data, the metadata, and the added interaction control data,

wherein the output data stream is sent to the decoder, wherein the presence of the interaction control data in the output data stream enables said decoder to identify that an interaction has happened.

16. A non-transitory digital storage medium storing a computer program configured to be executed by a computer to perform the method for processing packetized audio data, the method comprising:

receiving encoded audio data comprising audio objects and metadata related to the audio objects;

parsing the metadata to determine an audio data manipulation possibility of the audio objects;

receiving an interaction input and generating, from the interaction input, interaction control data related to the audio data manipulation possibility for manipulating the audio objects externally from a decoder; and

acquiring the interaction control data and the encoded audio data and the metadata and generating an output data stream by adding an additional data portion comprising the interaction control data to the output data stream, wherein the original metadata and the encoded audio data are not modified, such that the output data stream being again a valid encoded audio stream comprising the still encoded audio data, the metadata, and the added interaction control data,

wherein the output data stream is sent to the decoder, wherein the presence of the interaction control data in the output data stream enables said decoder to identify that an interaction has happened.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2020
From: SCHREINER, STEPHAN; NEUKAM, SIMONE; FUCHS, HARALD; PLOGSTIES, JAN; DOEHLA, STEFAN
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 052656/0970 →
Priority Claims (1)
EP 14170416 · May 28, 2014 · regional
Continuity (3)
Continuation 15357640 · Nov 21, 2016
Continuation PCTEP2015056768 · Mar 27, 2015
Related Publication 20200314505A1 · Oct 1, 2020
Cited By (1)
US 12,279,007