Audio processing techniques for semantic audio recognition and report generation
View Patent ↗Example apparatus, articles of manufacture and methods to determine semantic audio information for audio are disclosed. Example methods include extracting a plurality of audio features from the audio, at least one of the plurality of audio features including at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature. Example methods also include comparing the plurality of audio features to a plurality of stored audio feature ranges having tags associated therewith. Example methods further include determining a set of ranges of the plurality of stored audio feature ranges having closest matches to the plurality of audio features, a tag associated with the set of ranges having the closest matches to be used to determine the semantic audio information for the audio.
1. An apparatus to determine semantic audio information for audio, the apparatus comprising:
memory including computer readable instructions; and
a processor to execute the computer readable instructions to:
extract a plurality of audio features from the audio, at least one of the plurality of audio features including at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature;
compare the plurality of audio features to a plurality of stored audio feature ranges having tags associated therewith; and
determine a set of ranges of the plurality of stored audio feature ranges having closest matches to the plurality of audio features, a tag associated with the set of ranges having the closest matches to be used to determine the semantic audio information for the audio, wherein the tag is associated with at least one of an audio timbre range, a beat range, a loudness range or a spectral histogram range.
2. The apparatus of claim 1 , wherein the tag is associated with timber and the set of ranges includes a range for a mean of a spectral centroid, a range for a variance of the spectral centroid, and a range of a percentage of low/high energy frames.
3. The apparatus of claim 1 , wherein the tag is associated with beat and the set of ranges includes a range for an amplitude of peaks in a beat histogram, a range for a periods of peaks in the beat histogram, and a range for a ratio between a peak and a sum of all peaks in the beat histogram.
4. The apparatus of claim 3 , wherein the tag is associated with pitch and the set of ranges includes a range for an amplitude of prominent peaks in a pitch histogram, and a range for periods of peaks in the pitch histogram, wherein the pitch histogram is on a full semitone scale or an octave independent scale.
5. The apparatus of claim 1 , wherein (1) the harmonic feature includes at least one of a pitch, a tonality, a pitch class profile, harmonic changes, a main pitch class, an octave range of dominant pitch, a main tonal interval relation, or an overall pitch strength of at least some of the audio, (2) the rhythmic feature includes at least one of a rhythmic structure, a beat period, a rhythmic fluctuation, or an average tempo for at least some of the audio, (3) the spectral feature includes at least one of a spectral centroid, a spectral rolloff, a spectral flux, a spectral flatness measure, a spectral crest factor, Mel-frequency cepstral coefficients, Daubechies wavelet coefficients, a spectral dissonance, a spectral irregularity, or a spectral inharmonicity of at least some of the audio, and (4) the temporal feature includes at least one of amplitude, power, or zero crossing of at least some of the audio.
6. An article of manufacture comprising computer readable instructions that, when executed, cause a computing device to at least:
extract a plurality of audio features from the audio, at least one of the plurality of audio features including at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature;
compare the plurality of audio features to a plurality of stored audio feature ranges having tags associated therewith; and
determine a set of ranges of the plurality of stored audio feature ranges having closest matches to the plurality of audio features, a tag associated with the set of ranges having the closest matches to be used to determine the semantic audio information for the audio, wherein the tag is associated with at least one of an audio timbre range, a beat range, a loudness range or a spectral histogram range.
7. The article of manufacture of claim 6 , wherein the tag is associated with timber and the set of ranges includes a range for a mean of a spectral centroid, a range for a variance of the spectral centroid, and a range of a percentage of low/high energy frames.
8. The article of manufacture of claim 6 , wherein the tag is associated with beat and the set of ranges includes a range for an amplitude of peaks in a beat histogram, a range for a periods of peaks in the beat histogram, and a range for a ratio between a peak and a sum of all peaks in the beat histogram.
9. The article of manufacture of claim 6 , wherein the tag is associated with pitch and the set of ranges includes a range for an amplitude of prominent peaks in a pitch histogram, and a range for periods of peaks in the pitch histogram, wherein the pitch histogram is on a full semitone scale or an octave independent scale.
10. The article of manufacture of claim 6 , wherein (1) the harmonic feature includes at least one of a pitch, a tonality, a pitch class profile, harmonic changes, a main pitch class, an octave range of dominant pitch, a main tonal interval relation, or an overall pitch strength of at least some of the audio, (2) the rhythmic feature includes at least one of a rhythmic structure, a beat period, a rhythmic fluctuation, or an average tempo for at least some of the audio, (3) the spectral feature includes at least one of a spectral centroid, a spectral rolloff, a spectral flux, a spectral flatness measure, a spectral crest factor, Mel-frequency cepstral coefficients, Daubechies wavelet coefficients, a spectral dissonance, a spectral irregularity, or a spectral inharmonicity of at least some of the audio, and (4) the temporal feature includes at least one of amplitude, power, or zero crossing of at least some of the audio.
11. A method to determine semantic audio information for audio, the method comprising:
extracting, by executing an instruction with a processor, a plurality of audio features from the audio, at least one of the plurality of audio features including at least one of a temporal feature, a spectral feature, a harmonic feature, or a rhythmic feature;
comparing, by executing an instruction with the processor, the plurality of audio features to a plurality of stored audio feature ranges having tags associated therewith; and
determining, by executing an instruction with the processor, a set of ranges of the plurality of stored audio feature ranges having closest matches to the plurality of audio features, a tag associated with the set of ranges having the closest matches to be used to determine the semantic audio information for the audio, wherein the tag is associated with at least one of an audio timbre range, a beat range, a loudness range or a spectral histogram range.
12. The method of claim 11 , wherein the tag is associated with timber and the set of ranges includes a range for a mean of a spectral centroid, a range for a variance of the spectral centroid, and a range of a percentage of low/high energy frames.
13. The method of claim 11 , wherein the tag is associated with beat and the set of ranges includes a range for an amplitude of peaks in a beat histogram, a range for a periods of peaks in the beat histogram, and a range for a ratio between a peak and a sum of all peaks in the beat histogram.
14. The method of claim 11 , wherein the tag is associated with pitch and the set of ranges includes a range for an amplitude of prominent peaks in a pitch histogram, and a range for periods of peaks in the pitch histogram, wherein the pitch histogram is on a full semitone scale or an octave independent scale.
15. The method of claim 11 , wherein (1) the harmonic feature includes at least one of a pitch, a tonality, a pitch class profile, harmonic changes, a main pitch class, an octave range of dominant pitch, a main tonal interval relation, or an overall pitch strength of at least some of the audio, (2) the rhythmic feature includes at least one of a rhythmic structure, a beat period, a rhythmic fluctuation, or an average tempo for at least some of the audio, (3) the spectral feature includes at least one of a spectral centroid, a spectral rolloff, a spectral flux, a spectral flatness measure, a spectral crest factor, Mel-frequency cepstral coefficients, Daubechies wavelet coefficients, a spectral dissonance, a spectral irregularity, or a spectral inharmonicity of at least some of the audio, and (4) the temporal feature includes at least one of amplitude, power, or zero crossing of at least some of the audio.