Method and system for detecting sound event liveness using a microphone array
A method performed by an electronic device in a room. The method performs an enrollment process in which a spatial profile of a location of an artificial sound source is created and performs an identification process that determines whether a sound event within the room is produced by the artificial sound source by 1) capturing the sound event using a microphone array and 2) determining a likelihood that the sound event occurred at the location of the artificial sound source.
1. A method comprising:
capturing, using a microphone array, a sound event produced by a sound source within an environment of the microphone array as a first plurality of microphone signals;
producing, using a machine learning (ML) model, a spatial profile that identifies the sound source and a location within the environment at which the sound source is located based on the first plurality of microphone signals;
capturing, using the microphone array, a subsequent sound event within the environment as a second plurality of microphone signals;
determining whether the subsequent sound event originated from the location within the environment based on the second plurality of microphone signals and the spatial profile; and
responsive to determining that the sound event originated from the location, identifying the sound source based on the second plurality of microphone signals.
2. The method of claim 1 , wherein determining whether the subsequent sound event originated from the location comprises comparing spatial content of the second plurality of microphone signals and the spatial profile.
3. The method of claim 2 , wherein the spatial content comprises a direction of arrival (DoA) of the sound event with respect to the microphone array, wherein determining whether the subsequent sound event originated from the location comprises determining that the DoA matches at least a portion of the spatial profile based on the comparison.
4. The method of claim 1 , wherein identifying the sound source comprises:
extracting a spectral feature of the subsequent sound event from one or more microphone signals of the second plurality of microphone signals; and
comparing the extracted spectral feature with a stored spectral feature associated with the sound source.
5. The method of claim 1 further comprising extracting, from at least one microphone signal of the first plurality of microphone signals, 1) at least one spatial feature of the sound event that indicates the location of the sound source of the sound event with respect to the microphone array, and 2) at least one spectral feature of the sound event, wherein the spatial profile is produced as output of the ML model based on input of the at least one spatial feature and the at least one spectral feature.
6. The method of claim 1 , wherein the capturing of the sound event and the producing of the spatial profile occur during an enrollment process performed by an electronic device for the sound source, and the capturing of the subsequent sound event, determining, and identifying occur during an identification process subsequently performed by the electronic device.
7. The method of claim 1 , wherein the microphone array is a part of a smart speaker.
8. An electronic device, comprising:
a microphone array;
at least one processor; and
memory having instructions stored therein which when executed by the at least one processor causes the electronic device to:
capture, using the microphone array, a sound event produced by a sound source within an environment of the electronic device as a first plurality of microphone signals;
produce, using a machine learning (ML) model, a spatial profile that identifies the sound source and a location within the environment at which the sound source is located based on the first plurality of microphone signals;
capture, using the microphone array, a subsequent sound event within the environment as a second plurality of microphone signals;
determine whether the subsequent sound event originated from the location within the environment based on the second plurality of microphone signals and the spatial profile; and
responsive to determining that the sound event originated from the location, identify the sound source based on the second plurality of microphone signals.
9. The electronic device of claim 8 , wherein the instructions to determine whether the subsequent sound event originated from the location comprises instructions to compare spatial content of the second plurality of microphone signals and the spatial profile.
10. The electronic device of claim 9 , wherein the spatial content comprises a direction of arrival (DoA) of the sound event with respect to the microphone array, wherein the instructions to determine whether the subsequent sound event originated from the location comprises instructions to determine that the DoA matches at least a portion of the spatial profile based on the comparison.
11. The electronic device of claim 8 , wherein the instructions to identify the sound source comprises instructions to:
extract a spectral feature of the subsequent sound event from one or more microphone signals of the second plurality of microphone signals; and
compare the extracted spectral feature with a stored spectral feature associated with the sound source.
12. The electronic device of claim 8 , wherein the memory has further instructions to extract, from at least one microphone signal of the first plurality of microphone signals, 1) one or more spatial features of the sound event that indicates the location of the sound source of the sound event with respect to the microphone array, and 2) one or more spectral features of the sound event, wherein the spatial profile is produced as output of the ML model based on input of the one or more spatial features and the one or more spectral features.
13. The electronic device of claim 8 , wherein the capturing of the sound event and the producing of the spatial profile occur during an enrollment process performed by the electronic device for the sound source, and the capturing of the subsequent sound event, determining, and identifying occur during a subsequent identification process for the sound source.
14. The electronic device of claim 8 is a smart speaker.
15. Processing circuitry of an electronic device that is configured to:
capture, using a microphone array, a sound event produced by a sound source within an environment of the electronic device as a first plurality of microphone signals;
produce, using a machine learning (ML) model, a spatial profile that identifies the sound source and a location within the environment at which the sound source is located based on the first plurality of microphone signals;
capture, using the microphone array, a subsequent sound event within the environment as a second plurality of microphone signals;
determine whether the subsequent sound event originated from the location within the environment based on the second plurality of microphone signals and the spatial profile; and
responsive to determining that the sound event originated from the location, identify the sound source based on the second plurality of microphone signals.
16. The processing circuitry of claim 15 , wherein the processing circuitry determines whether the subsequent sound event originated from the location by comparing spatial content of the second plurality of microphone signals and the spatial profile.
17. The processing circuitry of claim 16 , wherein the spatial content comprises a direction of arrival (DoA) of the sound event with respect to the microphone array, wherein the processing circuitry determines whether the subsequent sound event originated from the location by determining that the DoA matches at least a portion of the spatial profile based on the comparison.
18. The processing circuitry of claim 15 , wherein the processing circuitry identifies the sound source by:
extracting a spectral feature of the subsequent sound event from one or more microphone signals of the second plurality of microphone signals; and
comparing the extracted spectral feature with a stored spectral feature associated with the sound source.
19. The processing circuitry of claim 15 , wherein the processing circuitry captures the sound event and produces the spatial profile during an enrollment process for the sound source, and the processing circuitry captures of the subsequent sound event, determines, and identifies during a subsequent identification process for the sound source.
20. The processing circuitry of claim 15 , wherein the electronic device is a smart speaker.