Automatically selecting a sound recognition model for an environment based on audio data and image data associated with the environment
A system for automatically selecting a sound recognition model for an environment based on audio data and image data associated with the environment. The system includes a camera, a microphone, a memory including a plurality of sound recognition models, and an electronic processor. The electronic processor is configured to receive the audio data associated with the environment from the microphone, receive the image data associated with the environment from the camera, and determine one or more characteristics of the environment based on the audio data and the image data. The electronic processor is also configured to select the sound recognition model from the plurality of sound recognition models based on the one or more characteristics of the environment, receive additional audio data associated with the environment from the microphone, and analyze the additional audio data using the sound recognition model to perform a sound recognition task.
1. A system for automatically selecting a sound recognition model for an environment based on audio data and image data associated with the environment, the system comprising:
a camera;
a microphone;
a memory including a plurality of sound recognition models; and
an electronic processor configured to
receive, via an input device, a selection of one or more sound recognition tasks;
receive the audio data associated with the environment from the microphone;
receive the image data associated with the environment from the camera;
determine one or more characteristics of the environment based on the audio data and the image data;
for each of the one or more sound recognition tasks selected, select the sound recognition model from the plurality of sound recognition models based on the one or more characteristics of the environment and the selected sound recognition task;
receive additional audio data associated with the environment from the microphone; and
analyze the additional audio data using the sound recognition model to perform a sound recognition task, wherein the sound recognition task includes generating a prediction regarding the additional audio data.
2. The system according to claim 1 , wherein the one or more characteristics include one selected from a group consisting of a depth map of the environment, an acoustic impulse response associated with the environment, a reverberation time associated with the environment, an acoustic property of a surface included in the environment, an acoustic absorption coefficient of a surface included in the environment, a signal-to-noise ratio associated with the environment, a direct-to-reverberant ratio associated with the environment, a clarity index associated with the environment, a dimensional measurement of the environments, and an acoustic scene.
3. The system according to claim 1 , wherein the electronic processor is configured to determine one or more characteristics of the environment based on the audio data and the image data using one or more deep learning models.
4. The system according to claim 1 , wherein each of the plurality of sound recognition models associated with an environment and is trained to perform a sound recognition task.
5. The system according to claim 1 , wherein the electronic processor is configured to select a sound recognition model from the plurality of sound recognition models based on the one or more characteristics of the environment using a heuristic model.
6. The system according to claim 1 , wherein the system includes a server and the electronic processor is further configured to
send the one or more characteristics of the environment to the server to be used by the server to retrain one or more sound recognition models the plurality of sound recognition models.
7. The system according to claim 6 , wherein the electronic processor is further configured to
receive the one or more retrained sound recognition models from the server.
8. The system according to claim 1 , wherein the system includes an electronic device installed in the environment and the camera, the microphone, the memory including the plurality of trained sound recognition models, and the electronic processor are included in the electronic device.
9. A method for automatically selecting a sound recognition model for an environment based on audio data and image data associated with the environment, the method comprising:
receiving, via an input device, a selection of one or more sound recognition tasks;
receiving the audio data associated with the environment from a microphone;
receiving the image data associated with the environment from a camera;
determining one or more characteristics of the environment based on the audio data and the image data;
for each of the one or more sound recognition tasks selected, selecting the sound recognition model from a plurality of sound recognition models based on the one or more characteristics of the environment and the selected sound recognition task;
receiving additional audio data associated with the environment from the microphone; and
analyzing the additional audio data using the sound recognition model to perform a sound recognition task, wherein the sound recognition task includes generating a prediction regarding the additional audio data.
10. The method according to claim 9 , wherein the one or more characteristics include one selected from a group consisting of a depth map of the environment, an acoustic impulse response associated with the environment, a reverberation time associated with the environment, an acoustic property of a surface included in the environment, an acoustic absorption coefficient of a surface included in the environment, a signal-to-noise ratio associated with the environment, a direct-to-reverberant ratio associated with the environment, a clarity index associated with the environment, a dimensional measurement of the environment, and an acoustic scene.
11. The method according to claim 9 , wherein determining one or more characteristics of the environment based on the audio data and the image data includes using one or more deep learning models to determine the one or more characteristics of the environment based on the audio data and the image data.
12. The method according to claim 9 , wherein each of the plurality of sound recognition models associated with an environment and is trained to perform a sound recognition task.
13. The method according to claim 9 , wherein selecting a sound recognition model from the plurality of sound recognition models based on the one or more characteristics of the environment includes using a heuristic model to selecting the trained sound recognition model from the plurality of sound recognition models based on the one or more characteristics of the environment.
14. The method according to claim 9 , the method further comprising
sending the one or more characteristics of the environment to a server to be used by the server to retrain one or more sound recognition models of the plurality of sound recognition models.
15. The method according to claim 14 , the method further comprising
receiving the one or more retrained sound recognition models from the server.
16. A system for automatically selecting a sound recognition model for an environment based on image data associated with the environment, the system comprising:
a camera;
a microphone;
a memory including a plurality of sound recognition models; and
an electronic processor configured to
receive, via an input device, a selection of one or more sound recognition tasks;
receive the image data associated with the environment from the camera;
determine one or more characteristics of the environment based on the image data;
for each of the one or more sound recognition tasks selected, select the sound recognition model from the plurality of sound recognition models based on the one or more characteristics of the environment and the selected sound recognition task;
receive audio data associated with the environment from the microphone; and
analyze the audio data using the sound recognition model to perform a sound recognition task, wherein a sound recognition task includes generating a prediction regarding the audio data.