Acoustic event detection
Techniques for reducing occurrences of cross-triggering event types not represented in audio data and false detection of event types are described. Different event types, such as a hand clap event type and a door knock event type may have substantially similar audio characteristics, and if one event type of such event types is represented in audio data, then event detection processing of that audio data may lead to detection of event types not represented in the audio data. Example embodiments involve training a model configured to detect multiple event types to enforce mutual exclusivity between different event type pairs or sets of the multiple event types. The model is trained to enforce mutual exclusivity using a regularizer function and a weight parameter to reduce any positive detection scores of event types not represented in received audio. Similar techniques may be applied to models for object detection using image data.
1 . A computer-implemented method comprising:
receiving training audio data representing an audio segment and ground truth data corresponding to labels for a plurality of first acoustic events including at least a first acoustic event and a second acoustic event;
generating, using a machine learning (ML) model, a first score corresponding to the first acoustic event and a second score corresponding to the second acoustic event;
based at least in part on the ground truth data indicating that the first acoustic event is represented in the audio segment and the second acoustic event is not represented in the audio segment, determining a regularizer function to enforce mutual exclusivity between the first acoustic event and the second acoustic event, wherein:
the regularizer function corresponds to adjusting a score corresponding to the second acoustic event to indicate a lower likelihood of the second acoustic event being represented in the audio segment;
determining first audio characteristic data corresponding to the first acoustic event is similar to second audio characteristic data corresponding to the second acoustic event;
determining, based at least in part on a similarity between the first audio characteristic data and the second audio characteristic data, a first weight parameter corresponding to a first pair of event types comprising the first acoustic event and the second acoustic event; and
training the ML model using the training audio data, the regularizer function, and the first weight parameter, to determine an updated ML model configured to determine, for the second acoustic event, an updated second score lower than the second score.
2 . The computer-implemented method of claim 1 , further comprising:
generating, using the first machine learning (ML) model, a third score corresponding to a third acoustic event;
determining the ground truth data indicates that the third acoustic event is not represented in the audio segment;
determining, based at least in part on a similarity between the first audio characteristic data and third audio characteristic data corresponding to the third acoustic event, a second weight parameter corresponding to a second pair of event types comprising the first acoustic event and the third acoustic event, wherein the second weight parameter is different from first weight parameter; and
training the ML model using the training audio data, the regularizer function, and the second weight parameter, to determine the updated ML model configured to determine, for the third acoustic event, an updated third score lower than the third score,
wherein the regularizer function is further determined to enforce mutual exclusivity between the first acoustic event and the third acoustic event.
3 . The computer-implemented method of claim 1 , wherein determining the regularizer function further comprises:
determining a third score by reducing the second score from a predetermined value; and
determining a fourth score based on a product of the first score and the third score, wherein the updated second score is based at least in part on the fourth score.
4 . The computer-implemented method of claim 1 , further comprising:
generating, using the ML model, a third score corresponding to a third acoustic event;
determining the ground truth data indicates that the third acoustic event is not represented in the audio segment; and
based at least in part on the third acoustic event being represented in the training audio data, refraining from determining a third weight parameter corresponding to a third pair of event types comprising the first acoustic event and the third acoustic event,
wherein the regularizer function is not configured to enforce mutual exclusivity between the first acoustic event and the third acoustic event.
5 . A computer-implemented method comprising:
receiving first audio data;
generating, using the first audio data and a machine learning (ML) model configured to detect occurrence of one or more event types in audio data, a first detection score corresponding to a first event type and a second detection score corresponding to a second event type, the ML model being trained with a regularizer function and a first weight parameter to generate the second detection score, wherein:
the regularizer function corresponds to adjusting a score corresponding to the second event type to indicate a lower likelihood of the second event type being represented in the first audio data, and
the first weight parameter is determined based at least in part on a similarity between the first event type and the second event type;
determining that the first detection score satisfies a threshold detection score; and
in response to determining the first detection score satisfies the threshold detection score, generating output data representing detection of the first event type.
6 . The computer-implemented method of claim 5 , wherein the ML model is trained by:
determining first audio characteristic data corresponding to the first event type is similar to second audio characteristic data corresponding to the second event type;
determining the first weight parameter based on a similarity between the first audio characteristic data and the second audio characteristic data; and
determining, based on the first weight parameter and the regularizer function, a first amount to reduce the score to generate the second detection score.
7 . The computer-implemented method of claim 6 , further comprising:
generating, using a training ML model and training audio data, a third detection score corresponding to a third event type, wherein the third detection score indicates a lower likelihood of the third event type being represented in the training audio data than a fourth detection score corresponding to the third event type, and wherein the third detection score is less than the fourth detection score by a second amount; and
determining a second weight parameter based on a similarity between the third event type and the first event type, wherein the second amount is determined based on the second weight parameter.
8 . The computer-implemented method of claim 5 , wherein the ML model is trained by:
operating the regularizer function to enforce mutual exclusivity between the first event type and the second event type.
9 . The computer-implemented method of claim 8 , further comprising:
determining a first weight parameter corresponding to a first set of event types comprising the first event type and the second event type,
wherein operating the regularizer function further comprises using the first weight parameter.
10 . The computer-implemented method of claim 9 , further comprising:
determining first audio characteristic data corresponding to the first event type and second audio characteristic data corresponding to the second event type; and
determining the first weight parameter based at least in part on a similarity between the first audio characteristic data and the second audio characteristic data.
11 . The computer-implemented method of claim 9 , further comprising:
performing at least one training iteration using a training ML model, training audio data, the regularizer function, and the first weight parameter, to determine the ML model configured to determine, for the second event type, a third detection score indicating a lower likelihood of the second event type being represented in the training audio data than a fourth detection score corresponding to the second event type.
12 . The computer-implemented method of claim 5 , wherein the ML model is trained by:
generating, using a training ML model and training audio data, a third detection score corresponding to a third event type;
determining that ground truth data indicates that the third event type is represented in the training audio data; and
based at least in part on the third event type being represented in the training audio data, refraining from determining a weight parameter corresponding to a set of event types comprising the first event type and the third event type,
wherein the regularizer function is not configured to enforce mutual exclusivity between the first event type and the third event type.
13 . A system comprising:
at least one processor; and
at least one memory including instructions that, when executed by the at least one processor, cause the system to:
receive first audio data;
generate, using the first audio data and a machine learning (ML) model configured to detect occurrence of one or more event types in audio data, a first detection score corresponding to a first event type and a second detection score corresponding to a second event type, the ML model being trained with a regularizer function and a first weight parameter to generate the second detection score, wherein:
the regularizer function corresponds to adjusting a score corresponding to the second event type to indicate a lower likelihood of the second event type being represented in the first audio data, and
the first weight parameter is determined based at least in part on a similarity between the first event type and the second event type;
determine that the first detection score satisfies a threshold detection score; and
in response to determining the first detection score satisfies the threshold detection score, generate output data representing detection of the first event type.
14 . The system of claim 13 , wherein the ML model is trained by:
determining first audio characteristic data corresponding to the first event type is similar to second audio characteristic data corresponding to the second event type;
determining the first weight parameter based on a similarity between the first audio characteristic data and the second audio characteristic data; and
determining, based on the first weight parameter and the regularizer function, a first amount to reduce the score to generate the second detection score.
15 . The system of claim 14 , wherein the ML model is further trained by:
generating, using a training ML model and training audio data, a third detection score corresponding to a third event type, wherein the third detection score indicates a lower likelihood of the third event type being represented in the training audio data than a fourth detection score corresponding to the third event type, and wherein the third detection score is less than the fourth detection score by a second amount; and
determining a second weight parameter based on a similarity between the third event type and the first event type, wherein the second amount is determined based on the second weight parameter.
16 . The system of claim 13 , wherein the ML model is trained by:
operating the regularizer function to enforce mutual exclusivity between the first event type and the second event type.
17 . The system of claim 16 , wherein the ML model is further trained by:
determining a first weight parameter corresponding to a first set of event types comprising the first event type and the second event type,
wherein operating the regularizer function further comprises using the first weight parameter.
18 . The system of claim 17 , wherein the ML model is further trained by:
determining first audio characteristic data corresponding to the first event type and second audio characteristic data corresponding to the second event type; and
determining the first weight parameter based at least in part on a similarity between the first audio characteristic data and the second audio characteristic data.
19 . The system of claim 17 , wherein the ML model is further trained by:
performing at least one training iteration using a training ML model, training audio data, the regularizer function, and the first weight parameter, to determine the ML model configured to determine, for the second event type, a third detection score indicating a lower likelihood of the second event type being represented in the training audio data than a fourth detection score corresponding to the second event type.
20 . The system of claim 13 , wherein the ML model is trained by:
generating, using a training ML model and training audio data, a third detection score corresponding to a third event type;
determining that ground truth data indicates that the third event type is represented in the training audio data; and
based at least in part on the third event type being represented in the training audio data, refraining from determining a weight parameter corresponding to a set of event types comprising the first event type and the third event type,
wherein the regularizer function is not configured to enforce mutual exclusivity between the first event type and the third event type.