Systems and methods for composition of audio content from multi-object audio
Embodiments are related to processing of one or more input audio feeds for generation of a target audio stream that includes at least one object of interest to a listener. In some embodiments, the target audio stream may exclusively or primarily include the sound of the object of interest to the listener, without including other persons. This allows a listener to focus on an object of his or her interest and not necessarily have to listen to the performances of other objects in the input audio feed. Some embodiments contemplate multiple audio feeds and/or with multiple objects of interest.
1. A method for composition of audio content comprising:
receiving an input audio feed including one or more objects distributed in multiple frames, wherein an object of interest in the one or more objects is identifiable based on a unique characteristic;
comparing of at least a portion of the input audio feed with the unique characteristic of the object of interest to detect matched frames that include the unique characteristic of the object of interest;
creating a timeline of the input audio feed;
annotating the timeline of the input audio feed at positions corresponding to positions of the matched frames;
generating a target audio stream having the object of interest by compositing the matched frames; and
partially eliminating objects that do not correspond to the object of interest from the target audio stream by suppressing, in the target audio stream, sounds of objects that do not correspond to the object of interest.
2. The method of claim 1 , wherein the annotating allows playing of the target audio stream at the positions of the matched frames.
3. The method of claim 1 , wherein the annotating allows selective seeking the positions of the matched frames, without generation of the target audio stream.
4. The method of claim 1 , wherein a user selects the object of interest from the timeline.
5. The method of claim 1 , wherein the object of interest is included in a user interface configured to enable a user selection of a set of objects of interest comprising the object of interest.
6. The method of claim 5 , wherein the object of interest is identified, at the user interface, using an identifier comprising one or more of a name, a role, a position in an event, a family, or an organization, the identifier being selectable via the user interface.
7. The method of claim 5 , wherein the object of interest is identified, at the user interface, using an image.
8. The method of claim 1 , further comprising:
generating a reference set comprising the object of interest; and
using the reference set to train a machine learning model to identify the object of interest in input audio streams.
9. The method of claim 8 , further comprising:
applying the machine learning model to compare the at least a portion of the input audio feed with the unique characteristic of the object of interest to detect the matched frames.
10. A non-transitory computer-readable storage medium storing instructions configured for composition of audio content to perform a method comprising:
receiving an input audio feed including one or more objects distributed in multiple frames, wherein an object of interest in the one or more objects is identifiable based on a unique characteristic;
comparing at least a portion of the input audio feed with the unique characteristic of the object of interest to detect matched frames that include the unique characteristic of the object of interest;
creating a timeline of the input audio feed;
annotating the timeline of the input audio feed at positions corresponding to positions of the matched frames;
generating a target audio stream having the object of interest by compositing the matched frames; and
partially eliminating objects that do not correspond to the object of interest, from the target audio stream by suppressing, in the target audio stream, sounds of objects that do not correspond to the object of interest.
11. The computer-readable storage medium of claim 10 , wherein the annotating allows playing of the target audio stream at the positions of the matched frames.
12. The computer-readable storage medium of claim 10 , wherein the annotating allows selective seeking the positions of the matched frames, without generation of the target audio stream.
13. The computer-readable storage medium of claim 10 , wherein a user selects the object of interest from the timeline.
14. An apparatus for composition of audio content comprising:
a memory;
one or more processors electronically coupled to the memory and configured for:
receiving an input audio feed including one or more objects distributed in multiple frames, wherein an object of interest in the one or more objects is identifiable based on a unique characteristic;
comparing at least a portion of the input audio feed with the unique characteristic of the object of interest to detect matched frames that include the unique characteristic of the object of interest;
creating a timeline of the input audio feed; and
annotating the timeline of the input audio feed at positions corresponding to positions of the matched frames;
generating a target audio stream having the object of interest by compositing the matched frames; and
partially eliminating objects that do not correspond to the object of interest, from the target audio stream by suppressing, in the target audio stream, sounds of objects that do not correspond to the object of interest.
15. The apparatus of claim 14 , wherein the annotating allows playing of the target audio stream at the positions of the matched frames.
16. The apparatus of claim 14 , wherein the annotating allows selective seeking the positions of the matched frames, without generation of the target audio stream.
17. The apparatus of claim 14 , wherein a user selects the object of interest from the timeline.