Voice detection optimization using sound metadata
Systems and methods for optimizing voice detection via a network microphone device are disclosed herein. In one example, individual microphones of a network microphone device detect sound. The sound data is captured in a first buffer and analyzed to detect a trigger event. Metadata associated with the sound data is captured in a second buffer and provided to at least one network device to determine at least one characteristic of the detected sound based on the metadata. The network device provides a response that includes an instruction, based on the determined characteristic, to modify at least one performance parameter of the NMD. The NMD then modifies the at least one performance parameter based on the instruction.
1. A network microphone device (NMD) comprising:
one or more processors;
one or more microphones; and
data storage having instructions thereon that, when executed by the one or more processors, cause the NMD to perform operations comprising:
detecting sound via the one or more microphones;
capturing sound data based on the detected sound, wherein the sound data includes a voice input;
analyzing the sound data to detect a wake word;
capturing metadata associated with the sound data, wherein the voice input is not derivable from the metadata;
transmitting the sound data to one or more remote computing devices to determine an intent based on the voice input;
transmitting the metadata to at least one of the one or more remote computing devices to determine at least one characteristic of the detected sound based on the metadata;
after transmitting the metadata, receiving a response from the at least one of the one or more remote computing devices, wherein the response includes an instruction, based on the determined characteristic, to modify at least one performance parameter of the NMD;
modifying the at least one performance parameter based on the instruction, wherein the modifying comprises at least one of:
adjusting a fixed gain of the NMD;
adjusting a wake-word-detection sensitivity parameter of the NMD;
adjusting a noise-reduction parameter of the NMD;
adjusting an acoustic echo cancellation parameter of the NMD;
adjusting a spatial processing algorithm of the NMD;
adjusting a localization algorithm of the NMD; or
disregarding input from a defective microphone; and
performing a command based on the determined intent.
2. The NMD of claim 1 , wherein transmitting the metadata occurs separately from transmitting the sound data.
3. The NMD of claim 1 , wherein transmitting the sound data comprises transmitting the sound data to one or more computing devices associated with a voice assistant service (VAS).
4. The NMD of claim 3 , wherein transmitting the metadata comprises transmitting the metadata to a remote evaluator distinct from the VAS.
5. The NMD of claim 1 , wherein the metadata comprises one or more of:
microphone frequency response data,
microphone spectral data,
acoustic echo cancellation (AEC) data,
echo return loss enhancement (ERLE) data,
arbitration data,
signal level data, or
direction detection data.
6. The NMD of claim 1 , wherein modifying the at least one performance parameter comprises adjusting one or more parameters of a multi-channel Wiener filter spatial processing algorithm of the NMD.
7. The NMD of claim 1 , wherein determining the at least one characteristic comprises comparing features of the metadata with averaged features of a sample population of sound metadata.
8. A tangible, non-transitory, computer-readable medium storing instructions that, when executed by one or more processors of a network microphone device (NMD), cause the NMD to perform operations comprising:
detecting sound via one or more microphones of the NMD;
capturing sound data based on the detected sound, wherein the sound data includes a voice input;
analyzing the sound data to detect a wake word;
capturing metadata associated with the sound data, wherein the voice input is not derivable from the metadata;
transmitting the sound data to one or more remote computing devices to determine an intent based on the voice input;
transmitting the metadata to at least one of the one or more remote computing devices to determine at least one characteristic of the detected sound based on the metadata;
after transmitting the metadata, receiving a response from the at least one of the one or more remote computing devices, wherein the response includes an instruction, based on the determined characteristic, to modify at least one performance parameter of the NMD;
modifying the at least one performance parameter based on the instruction, wherein the modifying comprises at least one of:
adjusting a fixed gain of the NMD;
adjusting a wake-word-detection sensitivity parameter of the NMD;
adjusting a noise-reduction parameter of the NMD;
adjusting an acoustic echo cancellation parameter of the NMD;
adjusting a spatial processing algorithm of the NMD;
adjusting a localization algorithm of the NMD; or
disregarding input from a defective microphone; and
performing a command based on the determined intent.
9. The computer-readable medium of claim 8 , wherein transmitting the metadata occurs separately from transmitting the sound data.
10. The computer-readable medium of claim 8 , wherein transmitting the sound data comprises transmitting the sound data to one or more computing devices associated with a voice assistant service (VAS).
11. The computer-readable medium of claim 10 , wherein transmitting the metadata comprises transmitting the metadata to a remote evaluator distinct from the VAS.
12. The computer-readable medium of claim 8 , wherein the metadata comprises one or more of:
microphone frequency response data,
microphone spectral data,
acoustic echo cancellation (AEC) data,
echo return loss enhancement (ERLE) data,
arbitration data,
signal level data, or
direction detection data.
13. The computer-readable medium of claim 8 , wherein modifying the at least one performance parameter comprises adjusting one or more parameters of a multi-channel Wiener filter spatial processing algorithm of the NMD.
14. The computer-readable medium of claim 8 , wherein determining the at least one characteristic comprises comparing features of the metadata with averaged features of a sample population of sound metadata.
15. A method performed by a network microphone device (NMD), the method comprising:
detecting sound via one or more microphones of the NMD;
capturing sound data based on the detected sound, wherein the sound data includes a voice input;
analyzing the sound data to detect a wake word;
capturing metadata associated with the sound data, wherein the voice input is not derivable from the metadata;
transmitting the sound data to one or more remote computing devices to determine an intent based on the voice input;
transmitting the metadata to at least one of the one or more remote computing devices to determine at least one characteristic of the detected sound based on the metadata;
after transmitting the metadata, receiving a response from the at least one of the one or more remote computing devices, wherein the response includes an instruction, based on the determined characteristic, to modify at least one performance parameter of the NMD;
modifying the at least one performance parameter based on the instruction, wherein the modifying comprises at least one of:
adjusting a fixed gain of the NMD;
adjusting a wake-word-detection sensitivity parameter of the NMD;
adjusting a noise-reduction parameter of the NMD;
adjusting an acoustic echo cancellation parameter of the NMD;
adjusting a spatial processing algorithm of the NMD;
adjusting a localization algorithm of the NMD; or
disregarding input from a defective microphone; and
performing a command based on the determined intent.
16. The method of claim 15 , wherein transmitting the metadata occurs separately from transmitting the sound data.
17. The method of claim 15 , wherein transmitting the sound data comprises transmitting the sound data to one or more computing devices associated with a voice assistant service (VAS).
18. The method of claim 17 , wherein transmitting the metadata comprises transmitting the metadata to a remote evaluator distinct from the VAS.
19. The method of claim 15 , wherein the metadata comprises one or more of:
microphone frequency response data,
microphone spectral data,
acoustic echo cancellation (AEC) data,
echo return loss enhancement (ERLE) data,
arbitration data,
signal level data, or
direction detection data.
20. The method of claim 15 , wherein modifying the at least one performance parameter comprises adjusting one or more parameters of a multi-channel Wiener filter spatial processing algorithm of the NMD.