Audio processing techniques for semantic audio recognition and report generation
Example methods, apparatus and articles of manufacture to determine semantic information for audio are disclosed. Example apparatus disclosed herein are to process an audio signal obtained by a media device to determine values of a plurality of features that are characteristic of the audio signal, compare the values of the plurality of features to a first template having corresponding first ranges of the plurality of features to determine a first score, the first template associated with first semantic information, compare the values of the plurality of features to a second template having corresponding second ranges of the plurality of features to determine a second score, the second template associated with second semantic information, and associate the audio signal with at least one of the first semantic information or the second semantic information based on the first score and the second score.
1. An apparatus to determine semantic information for audio, the apparatus comprising:
memory including computer readable instructions; and
a processor to execute the computer readable instructions to at least:
process a first frame of an audio signal obtained by a media device to determine first values of a plurality of features that are characteristic of the audio signal;
compare the first values of the plurality of features to a first template having corresponding first ranges of the plurality of features to determine a first score, the first template associated with first semantic information;
compare the first values of the plurality of features to a second template having corresponding second ranges of the plurality of features to determine a second score, the second template associated with second semantic information;
associate the first frame of the audio signal with at least one of the first semantic information or the second semantic information based on the first score and the second score;
process a second audio frame of the audio signal to determine second values of the plurality of features;
compare the second values of the plurality of features to the first template to determine a third score;
compare the second values of the plurality of features to the second template to determine a fourth score; and
associate the second audio frame of the audio signal with at least one of the first semantic information or the second semantic information based on the third score and the fourth score.
2. The apparatus of claim 1 , wherein the plurality of features includes at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature.
3. The apparatus of claim 2 , wherein the harmonic feature includes at least one of a pitch, a tonality, a pitch class profile, a harmonic change, a main pitch class, an octave range of dominant pitch, a main tonal interval relation, or an overall pitch strength of at least some of the audio signal.
4. The apparatus of claim 2 , wherein the rhythmic feature includes at least one of a rhythmic structure, a beat period, a rhythmic fluctuation, or an average tempo for at least some of the audio signal.
5. The apparatus of claim 2 , wherein the spectral feature includes at least one of a spectral centroid, a spectral rolloff, a spectral flux, a spectral flatness measure, a spectral crest factor, Mel-frequency cepstral coefficients, Daubechies wavelet coefficients, a spectral dissonance, a spectral irregularity, or a spectral inharmonicity of at least some of the audio signal.
6. The apparatus of claim 2 , wherein the temporal feature includes at least one of amplitude, power, or zero crossing of at least some of the audio signal.
7. The apparatus of claim 1 , wherein the first ranges include at least one of a first range for an audio timbre feature, a first range for a beat feature, a first range for a loudness feature or a first range for a spectral histogram feature, and the second ranges include at least one of a second range for an audio timbre feature, a second range for a beat feature, a second range for a loudness feature or a second range for a spectral histogram feature.
8. The apparatus of claim 1 , wherein:
the first semantic information includes at least one of a first genre descriptor, a first instrumentation descriptor, a first style descriptor, a first acoustical dynamics descriptor, or a first emotive descriptor; and
the second semantic information includes at least one of a second genre descriptor, a second instrumentation descriptor, a second style descriptor, a second acoustical dynamics descriptor, or a second emotive descriptor.
9. The apparatus of claim 1 , wherein respective ones of the first ranges are associated with corresponding first weights, and respective ones of the second ranges are associated with corresponding second weights.
10. An article of manufacture comprising computer readable instructions which, when executed, cause a processor to at least:
process a first frame of an audio signal obtained by a media device to determine first values of a plurality of features that are characteristic of the audio signal;
compare the first values of the plurality of features to a first template having corresponding first ranges of the plurality of features to determine a first score, the first template associated with first semantic information;
compare the first values of the plurality of features to a second template having corresponding second ranges of the plurality of features to determine a second score, the second template associated with second semantic information;
associate the first frame of the audio signal with at least one of the first semantic information or the second semantic information based on the first score and the second score;
process a second audio frame of the audio signal to determine second values of the plurality of features;
compare the second values of the plurality of features to the first template to determine a third score;
compare the second values of the plurality of features to the second template to determine a fourth score; and
associate the second audio frame of the audio signal with at least one of the first semantic information or the second semantic information based on the third score and the fourth score.
11. The article of manufacture of claim 10 , wherein the plurality of features includes at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature.
12. The article of manufacture of claim 10 , wherein the first ranges include at least one of a first range for an audio timbre feature, a first range for a beat feature, a first range for a loudness feature or a first range for a spectral histogram feature, and the second ranges include at least one of a second range for an audio timbre feature, a second range for a beat feature, a second range for a loudness feature or a second range for a spectral histogram feature.
13. The article of manufacture of claim 10 , wherein:
the first semantic information includes at least one of a first genre descriptor, a first instrumentation descriptor, a first style descriptor, a first acoustical dynamics descriptor, or a first emotive descriptor; and
the second semantic information includes at least one of a second genre descriptor, a second instrumentation descriptor, a second style descriptor, a second acoustical dynamics descriptor, or a second emotive descriptor.
14. The article of manufacture of claim 10 , wherein respective ones of the first ranges are associated with corresponding first weights, and respective ones of the second ranges are associated with corresponding second weights.
15. A method to determine semantic information for audio, the method comprising:
processing, by executing an instruction with a processor, a first frame of an audio signal obtained by a media device to determine first values of a plurality of features that are characteristic of the audio signal;
comparing, by executing an instruction with the processor, the first values of the plurality of features to a first template having corresponding first ranges of the plurality of features to determine a first score, the first template associated with first semantic information;
comparing, by executing an instruction with the processor, the first values of the plurality of features to a second template having corresponding second ranges of the plurality of features to determine a second score, the second template associated with second semantic information;
associating, by executing an instruction with the processor, the first frame of the audio signal with at least one of the first semantic information or the second semantic information based on the first score and the second score;
processing, by executing an instruction with the processor, a second audio frame of the audio signal to determine second values of the plurality of features;
comparing, by executing an instruction with the processor, the second values of the plurality of features to the first template to determine a third score;
comparing, by executing an instruction with the processor, the second values of the plurality of features to the second template to determine a fourth score; and
associating, by executing an instruction with the processor, the second audio frame of the audio signal with at least one of the first semantic information or the second semantic information based on the third score and the fourth score.
16. The method of claim 15 , wherein the plurality of features includes at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature.
17. The method of claim 15 , wherein the first ranges include at least one of a first range for an audio timbre feature, a first range for a beat feature, a first range for a loudness feature or a first range for a spectral histogram feature, and the second ranges include at least one of a second range for an audio timbre feature, a second range for a beat feature, a second range for a loudness feature or a second range for a spectral histogram feature.
18. The method of claim 15 , wherein:
the first semantic information includes at least one of a first genre descriptor, a first instrumentation descriptor, a first style descriptor, a first acoustical dynamics descriptor, or a first emotive descriptor; and
the second semantic information includes at least one of a second genre descriptor, a second instrumentation descriptor, a second style descriptor, a second acoustical dynamics descriptor, or a second emotive descriptor.
19. The method of claim 15 , wherein respective ones of the first ranges are associated with corresponding first weights, and respective ones of the second ranges are associated with corresponding second weights.