IP Library › Granted Patent US 12,271,815
Granted Patent B2
US 12,271,815 · App. 17/863,599 · Granted Apr 8, 2025

Systems and methods for false positive mitigation in impulsive sound detectors

Inventors: Luca Bondi (Pittsburgh, PA); Samarjit Das (Wexford, PA); Shabnam Ghaffarzadegan (Livermore, CA)
Assignee: Robert Bosch GmbH
G06N3/08G10L25/21G10L25/30G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,271,815
App. No.
17/863,599
Filed
Jul 13, 2022
Granted
Apr 8, 2025
Kind
B2
Art Unit
2658
USPC
704/232
Abstract

A method data augmentation includes receiving audio stream data associated with at least one impulse event, receiving a label associated with the audio stream data, and detecting, using an onset detector, at least one peak of the at least one impulse event. The method also includes extracting at least one positive sample of the audio stream data associated with the at least one impulse event. The method also includes applying, to the at least one positive sample, the label associated with the audio stream data and extracting at least one negative sample of the audio stream data associated with the at least one impulse event. The method also includes augmenting training data based on the at least one positive sample and the at least one negative sample and training at least one machine-learning model using the augmented training data.

Claims (48)

1. A computer-implemented method for training an impulse detection model, the method comprising, at a processor:

receiving audio stream data associated with at least one impulse event;

receiving a label associated with the audio stream data;

detecting, using an onset detector algorithm, at least one peak of the at least one impulse event;

extracting at least one positive sample of the audio stream data associated with the at least one impulse event, wherein the at least one positive sample includes a portion of the audio stream data associated with a first time value of the audio stream data;

labeling the at least one positive sample with the label associated with the audio stream data to yield at least one labeled positive sample;

extracting at least one negative sample of the audio stream data associated with the at least one impulse event, wherein the at least one negative sample does not include the portion of the audio stream data associated with the first time value of the audio stream data;

augmenting machine-learning training data based on the at least one labeled positive sample and the at least one negative sample; and

training, using the processor executing instructions stored on a memory, at least one machine-learning model with the augmented machine-learning training data, wherein the at least one machine-learning model includes an impulsive event detection machine-learning model and is configured to detect, at least, a class of sound associated with the at least one impulse event.

2. The computer-implemented method of claim 1 , wherein the at least one positive sample extends a first period before the first time value and a second period after the first time value.

3. The computer-implemented method of claim 1 , wherein the at least one negative sample includes a label different from the label associated with the audio stream data.

4. The computer-implemented method of claim 1 , wherein the at least one negative sample and the at least one positive sample do not overlap.

5. The computer-implemented method of claim 1 , wherein training the at least one machine-learning model with the augmented training data includes training the at least one machine-learning model with at least one backpropagation technique.

6. The computer-implemented method of claim 1 , further comprising further augmenting the machine-learning training data using at least one of a spectral masking, a spectral shift, at least one background sound, and at least one impulse response associated with an acoustic environment.

7. The computer-implemented method of claim 1 , wherein the onset detector algorithm is configured to detect the at least one peak using at least one of an energy envelop associated with the audio stream data, at least one energy floor associated with the audio stream data, at least one energy peak associated with the audio stream data, and at least one dynamic range associated with the audio stream data.

8. A system for training an impulsive event detection machine-learning model, the system comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to:

receive audio stream data associated with at least one impulse event;

receive a label associated with the audio stream data;

detect, using an onset detector algorithm, at least one peak of the at least one impulse event;

extract at least one positive sample of the audio stream data associated with the at least one impulse event, wherein the at least one positive sample includes a portion of the audio stream data associated with a first time value of the audio stream data;

label, the at least one positive sample, with the label associated with the audio stream data to yield at least one labeled positive sample;

extract at least one negative sample of the audio stream data associated with the at least one impulse event, wherein the at least one negative sample does not include the portion of the audio stream data associated with the first time value of the audio stream data;

augment machine-learning training data based on the at least one labeled positive sample and the at least one negative sample; and

train the impulsive event detection machine-learning model using the augmented machine-learning training data.

9. The system of claim 8 , wherein the at least one positive sample extends a first period before the first time value and a second period after the first time value.

10. The system of claim 8 , wherein the at least one negative sample includes a label different from the label associated with the audio stream data.

11. The system of claim 8 , wherein the at least one negative sample and the at least one positive sample do not overlap.

12. The system of claim 8 , wherein the instructions further cause the processor to detect, using the impulsive event detection machine-learning model, one or more impulse events associated with audio data provided as input to the impulsive event detection machine-learning model.

13. The system of claim 8 , wherein training the impulsive event detection machine-learning model using the augmented machine-learning training data includes training the impulsive event detection machine-learning model with at least one backpropagation technique.

14. The system of claim 8 , wherein the instructions further cause the processor to further augment the machine-learning training data using at least one of a spectral masking, a spectral shift, at least one background sound, and at least one impulse response associated with an acoustic environment.

15. The system of claim 8 , wherein the onset detector algorithm is configured to detect the at least one peak using at least one of an energy envelop associated with the audio stream data, at least one energy floor associated with the audio stream data, at least one energy peak associated with the audio stream data, and at least one dynamic range associated with the audio stream data.

16. An apparatus for impulsive event detection, the apparatus comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to:

receive audio stream data associated with at least one impulse event;

receive a label associated with the audio stream data;

detect, using an onset detector algorithm, at least one peak of the at least one impulse event, wherein the onset detector algorithm is configured to detect the at least one peak using at least one of an energy envelop associated with the audio stream data, at least one energy floor associated with the audio stream data, at least one energy peak associated with the audio stream data, and at least one dynamic range associated with the audio stream data;

extract at least one positive sample of the audio stream data associated with the at least one impulse event, wherein the at least one positive sample includes a portion of the audio stream data associated with a first time value of the audio stream data;

label, the at least one positive sample, with the label associated with the audio stream data to yield at least one labeled positive sample;

extract at least one negative sample of the audio stream data associated with the at least one impulse event, wherein the at least one negative sample does not include the portion of the audio stream data associated with the first time value of the audio stream data;

augment, using at least one of a spectral masking, a spectral shift, at least one background sound, and at least one impulse response associated with an acoustic environment, machine-learning training data based on the at least one labeled positive sample and the at least one negative sample;

train at least one machine-learning model using the augmented machine-learning training data; and

detect, using the at least one machine-learning model, one or more impulse events associated with audio data provided as input to the at least one machine-learning model.

17. The apparatus of claim 16 , wherein the at least one positive sample extends a first period before the first time value and a second period after the first time value.

18. The apparatus of claim 16 , wherein the at least one negative sample includes a label different from the label associated with the audio stream data.

19. The apparatus of claim 16 , wherein the at least one negative sample and the at least one positive sample do not overlap.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2022
From: BONDI, LUCA; DAS, SAMARJIT; GHAFFARZADEGAN, SHABNAM
To: ROBERT BOSCH GMBH
Reel/Frame 060493/0259 →
Continuity (1)
Related Publication 20240020526A1 · Jan 18, 2024
References Cited (14)
US 10296638B1 · Chen · 2019 [cited by examiner]
US 10937444B1 · Suendermann-Oeft · 2021 [cited by examiner]
US 20130272548A1 · Visser · 2013 [cited by examiner]
US 20200241552A1 · Leenayongwut · 2020 [cited by examiner]
US 20210065733A1 · Sallem · 2021 [cited by examiner]
US 20210136510A1 · Tang · 2021 [cited by examiner]
US 20210289168A1 · Gl?Ckert · 2021 [cited by examiner]
US 20220027672A1 · Xu · 2022 [cited by examiner]
US 20240020526A1 · Bondi · 2024 [cited by examiner]
CN 112183729A · 2021 [cited by applicant]
Li Juncheng et al: “A Light-Weight Multimodal Framework for Improved Environmental Audio Tagging”, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Apr. 15, 2018, pp. 6832-68… [cited by applicant]
Takahashi et al.: “Deep Convolutional Neural Networks and Data Augmentation for Acoustic Even Detection”, Dec. 8, 2016. [cited by applicant]
Li et al: “Learning to Self-Train for Semi-Supervised Few-Shot Classification Tat-Seng Chua”, Sep. 29, 2019. [cited by applicant]
Yu et al: “Few-Shot Sound Event Detection”, 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, May 4, 2020. [cited by applicant]