Accoustic context recognition using local binary pattern method and apparatus
Various exemplary aspects are directed to acoustic context recognition apparatuses and methods involving isolating and identifying context(s) of an acoustic environment. In one exemplary embodiment, source audio is converted into audio spectrograms, each spectrogram indicative of a period of time. The series of spectrograms are analyzed to identify audio patterns, over a period of time, which are indicative of an environmental context of the source audio. In many embodiments of the present disclosure, acoustic context recognition also includes comparing the identified audio patterns to known environmental contexts.
1. A method comprising:
receiving an audio signal at a microphone that is built in to an electronic device;
constructing an audio signal spectrogram, having adjacent pixels, indicative of the audio received at the microphone;
dividing the audio signal spectrogram into a plurality of blocks;
constructing a plurality of local binary patterns (LBP) based on a comparison of the adjacent pixels of the spectrogram;
creating, for each of the plurality of blocks of the spectrogram, a respective LBP histogram that is based on a number of times different LBPs occur in a corresponding block of the plurality of blocks;
identifying clusters of LBP histograms from the respective LBP histograms;
generating, for each of the clusters, a code word representing a corresponding cluster;
creating a codebook histogram based upon correspondence between the LBP histograms and the code words representing the clusters; and
classifying, using a machine learning model, the codebook histogram to identify environmental context that indicates a location of the electronic device at the time the audio signal was received at the microphone;
wherein the respective LBP histogram, for each of the plurality of blocks of the spectrogram, is indicative of an acoustic context of the audio input over a period of time;
wherein the audio signal spectrogram is a linear spectrum representation of the audio input over a frequency range and wherein each of the plurality of blocks for which an LBP histogram is created represents a particular sub-time over the period of time and a particular sub-frequency range that is within the frequency range of the audio signal.
2. The method of claim 1 , wherein the plurality of LBPs are indicative of pixel values, threshold transitions, and respective pixel locations in each of the plurality of blocks of the spectrogram.
3. The method of claim 1 , wherein creation of the codebook histogram further includes:
determining a cosine distance between the LBP histograms and the code words using the cosine distance between the LBP histograms and the code words, and
assigning the LBP histograms to code words in the codebook histogram based upon the cosine distance.
4. The method of claim 1 , further comprising the step of interpolating, for each of the LBP histograms, by using pixel values outside a respective block for each of the LBP histograms.
5. The method of claim 1 , wherein each respective LBP histogram is a histogram of a relationship between each of the plurality of blocks of the spectrogram as defined by a plurality of uniform configurations and a non-uniform configuration.
6. The method of claim 1 , wherein the plurality of blocks of the spectrogram are defined by blocks within 0 Hz-900 Hz frequency-band, blocks within 900 Hz-2000 Hz frequency-band, and blocks within 2000 Hz-8000 Hz frequency-band.
7. The method of claim 1 , wherein the LBP is defined by an algorithm:
LBP
P
,
R
=
∑
i
=
0
P
-
1
f
(
g
i
-
μ
)
2
P
,
f
(
x
)
=
{
1
,
x
≥
μ
0
,
x
<
μ
,
wherein g i is a neighboring pixel's value, μ is a mean over a particular block, P is a number of pixels in the particular block, R is a radius of a neighborhood and wherein coordinates of g i are R cos(2πi/P), sin(2πi/P).
8. The method of claim 1 , wherein the electronic device is a cellphone.
9. The method of claim 1 , wherein the electronic device is a mobile device.
10. A method comprising the steps of:
receiving an audio signal spectrogram indicative of an audio input;
dividing the audio signal spectrogram into a plurality of blocks;
constructing a plurality of local binary patterns (LBP) based on a comparison of adjacent pixels of the spectrogram;
creating, for each of the plurality of blocks of the spectrogram, a respective LBP histogram that is based on a number of times different LBPs occur in a corresponding block of the plurality of blocks;
identifying clusters of LBP histograms from the respective LBP histograms for each of the plurality of blocks of the spectrogram;
generating, based upon the clusters, a codebook containing a plurality of code words;
mapping the LBP histograms to the code words of the codebook to create a codebook histogram; and
training a machine learning algorithm using the LBP histograms as features therefore to identify environmental context that indicates a location of an electronic device at the time an audio signal is received at a microphone of the electronic device;
wherein the respective LBP histogram, for each of the plurality of blocks of the spectrogram, is indicative of an acoustic context of the audio input over a period of time;
wherein the audio signal spectrogram is a linear spectrum representation of the audio input over a frequency range and wherein each of the plurality of blocks for which an LBP histogram is created represents a particular sub-time over the period of time and a particular sub-frequency range that is within the frequency range of the audio signal.
11. The method of claim 10 , further comprising interpolating, for each of the LBP histograms, using pixel values outside a respective block for each of the LBP histograms.
12. The method of claim 10 , wherein each respective LBP histogram maps each of the plurality of blocks of the spectrogram into a plurality of uniform configurations and a non-uniform configuration.
13. The method of claim 10 , wherein the plurality of blocks of the spectrogram are defined by blocks within 0 Hz-900 Hz frequency-band, blocks within 900 Hz-2000 Hz frequency-band, and blocks within 2000 Hz-8000 Hz frequency-band.
14. The method of claim 10 , wherein the machine learning algorithm is within a support vector machine.
15. The method of claim 10 , wherein the LBP is defined by an algorithm:
LBP
P
,
R
=
∑
i
=
0
P
-
1
f
(
g
i
-
μ
)
2
P
,
f
(
x
)
=
{
1
,
x
≥
μ
0
,
x
<
μ
,
where gi is a neighboring pixel's value, μ is a mean over a particular block, P is a number of pixels in the particular block, R is a radius of a neighborhood and wherein the coordinates of gi are R cos(2πi/P), sin(2πi/P).
16. The method of claim 10 , wherein the electronic device is a cellphone.
17. The method of claim 10 , wherein the electronic device is a mobile device.