Open vs enclosed spatial environment classification for a mobile or wearable device using microphone and deep learning method
A method and apparatus for classifying a spatial environment as open or enclosed are provided. In the method and apparatus, one or more microphones detect ambient sound in a spatial environment and output an audio signal representative of the ambient sound. A processor determines a spatial environment impulse response (SEIR) for the audio signal and extracts one or more features of the SEIR. The processor classifies the spatial environment as open or enclosed based on the one or more features of the SEIR.
1. A method, comprising:
detecting, by one or more microphones, ambient sound in a spatial environment;
outputting, to a processor, an audio signal representative of the ambient sound;
obtaining a plurality of frames of the audio signal, the plurality of frames including a first frame and a second frame;
determining an energy of the first frame and an energy of the second frame; and
classifying, by the processor, a type of the spatial environment based on the energy of the first frame and the energy of the second frame.
2. The method of claim 1 , wherein classifying the type of the spatial environment based on the energy of the first frame and the energy of the second frame includes classifying the type of the spatial environment based on a ratio of the energy of the first frame to the energy of the second frame.
3. The method of claim 1 , comprising:
determining a spatial environment impulse response (SEIR) for the audio signal by at least:
selecting, from the plurality of frames, a set of frames having respective energy ratios that meet a criterion, the set of frames including at least one of the first frame and the second frame;
performing windowing on the set of frames to minimize phase;
determining a cepstrum for the set of frames; and
performing inverse exponential windowing on the set of frames.
4. The method of claim 3 , wherein classifying the type of the spatial environment based on the energy of the first frame and the energy of the second frame includes classifying the type of the spatial environment based on one or more features of the SEIR.
5. The method of claim 4 , comprising:
augmenting the one or more features of the SEIR with features extracted from Mel-Frequency Cepstral Coefficients (MFCCs), delta MFCCs or double delta MFCC to form a composite vector; and
classifying the spatial environment based on the composite vector.
6. The method of claim 1 , wherein the type of the spatial environment is an open spatial environment or an enclosed spatial environment.
7. The method of claim 1 , wherein the type of the spatial environment is at least one of: an office, home, mall, supermarket, street, stadium, beach or nature.
8. A device, comprising:
one or more microphones configured to:
detect ambient sound in a spatial environment; and
output an audio signal representative of the ambient sound; and
a processor configured to:
obtain a plurality of frames of the audio signal, the plurality of frames including a first frame and a second frame;
determine an energy of the first frame and an energy of the second frame; and
classify a type of the spatial environment based on the energy of the first frame and the energy of the second frame.
9. The device of claim 8 , wherein classifying the type of the spatial environment based on the energy of the first frame and the energy of the second frame includes classifying the type of the spatial environment based on a ratio of the energy of the first frame to the energy of the second frame.
10. The device of claim 8 , wherein the processor is configured to:
determine a spatial environment impulse response (SEIR) for the audio signal by at least:
selecting, from the plurality of frames, a set of frames having respective energy ratios that meet a criterion, the set of frames including at least one of the first frame and the second frame;
performing windowing on the set of frames to minimize phase;
determining a cepstrum for the set of frames; and
performing inverse exponential windowing on the set of frames.
11. The device of claim 10 , wherein classifying the type of the spatial environment based on the energy of the first frame and the energy of the second frame includes classifying the type of the spatial environment based on one or more features of the SEIR.
12. The device of claim 11 , wherein the processor is configured to:
augment the one or more features of the SEIR with features extracted from Mel-Frequency Cepstral Coefficients (MFCCs), delta MFCCs or double delta MFCC to form a composite vector; and
classify the spatial environment based on the composite vector.
13. The device of claim 8 , wherein the type of the spatial environment is an open spatial environment or an enclosed spatial environment.
14. The device of claim 8 , wherein the type of the spatial environment is at least one of: an office, home, mall, supermarket, street, stadium, beach or nature.
15. A system, comprising:
a processor; and
memory configured to store executable instructions that, when executed by the processor, cause the processor to:
receive an audio signal representative of ambient sound of a spatial environment;
obtain a plurality of frames of the audio signal, the plurality of frames including a first frame and a second frame;
determine an energy of the first frame and an energy of the second frame; and
classify a type of the spatial environment based on the energy of the first frame and the energy of the second frame.
16. The system of claim 15 , wherein classifying the type of the spatial environment based on the energy of the first frame and the energy of the second frame includes classifying the type of the spatial environment based on a ratio of the energy of the first frame to the energy of the second frame.
17. The system of claim 15 , wherein the processor is configured to:
determine a spatial environment impulse response (SEIR) for the audio signal by at least:
selecting, from the plurality of frames, a set of frames having respective energy ratios that meet a criterion, the set of frames including at least one of the first frame and the second frame;
performing windowing on the set of frames to minimize phase;
determining a cepstrum for the set of frames; and
performing inverse exponential windowing on the set of frames.
18. The system of claim 17 , wherein classifying the type of the spatial environment based on the energy of the first frame and the energy of the second frame includes classifying the type of the spatial environment based on one or more features of the SEIR.
19. The system of claim 18 , wherein the processor is configured to:
augment the one or more features of the SEIR with features extracted from Mel-Frequency Cepstral Coefficients (MFCCs), delta MFCCs or double delta MFCC to form a composite vector; and
classify the spatial environment based on the composite vector.
20. The system of claim 15 , wherein the type of the spatial environment is an open spatial environment or an enclosed spatial environment.