IP Library › Granted Patent US 11,074,927
Granted Patent B2
US 11,074,927 · App. 15/798,462 · Granted Jul 27, 2021

Acoustic event detection in polyphonic acoustic data

Inventors: Xiao Xing Liang (Beijing, CN); Ning Zhang (Beijing, CN); Yu Ling Zheng (Beijing, CN); Yu Chen Zhou (Beijing, CN)
Assignee: International Business Machines Corporation
G10L25/30G06N3/0445G06N3/08G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,074,927
App. No.
15/798,462
Granted
Jul 27, 2021
Kind
B2
Abstract

A computer implemented method, computer system and computer program product are provided for acoustic event detection in polyphonic acoustic data, according to the method, polyphonic acoustic data is inputted by one or more processing units into a trained neural network trained by labeled monophonic acoustic data, a first output from a hidden layer of the trained neural network is obtained by one or more processing units, and at least one acoustic classification of the polyphonic acoustic data is determined by one or more processing units based on the first output and a feature dictionary learnt from the trained neural network.

Claims (52)

1. A computer-implemented method comprising:

inputting, by one or more processing units, polyphonic acoustic data into a trained neural network trained by labeled monophonic acoustic data, wherein the trained neural network is trained by:

inputting, by one or more processing units, the labeled monophonic acoustic data into the trained neural network; and

obtaining, by one or more processing units, the trained neural network by using the label vectors of the labeled monophonic acoustic data as the ground truth of a second output of the trained neural network;

obtaining, by one or more processing units, a first output from a hidden layer of the trained neural network; and

determining, by one or more processing units, at least one acoustic classification of the polyphonic acoustic data based on the first output and a feature dictionary learnt from the trained neural network.

2. The method of claim 1 , wherein the determining comprises:

representing, by one or more processing units, the first output as a non-negative sparse combination of the dictionary;

computing, by one or more processing units, a membership function of the first output based on the non-negative sparse combination of the dictionary; and

determining, by one more processing units, an event of an acoustic classification occurred in the polyphonic acoustic data in response to a membership value of the membership function being larger than an assigned threshold.

3. The method of claim 1 , wherein the feature dictionary is learnt by:

inputting, by one or more processing units, the monophonic acoustic data into the trained neural network;

obtaining, by one or more processing units, a third output from a hidden layer of the trained neural network; and

learning, by one or more processing units, the feature dictionary based on the third output.

4. The method of claim 3 , wherein the learning is used to optimize an object function for the at least one acoustic classification.

5. The method of claim 1 , wherein the labeled monophonic acoustic data are denoted as feature vectors corresponding to a sound segment.

6. The method of claim 1 , wherein the hidden layer is the last hidden layer.

7. The method of claim 1 , wherein the trained neural network is a Recurrent Neural Network (RNN) with Long Short Term Memory (LSTM) neurons.

8. A computer system, comprising:

a processor;

a computer-readable memory coupled to the processor, the memory comprising instructions that when executed by the processor perform actions of:

inputting polyphonic acoustic data into a trained neural network trained by labeled monophonic acoustic data, wherein the trained neural network is trained by:

inputting the labeled monophonic acoustic data into the trained neural network; and

obtaining the trained neural network by using the label vectors of the labeled monophonic acoustic data as the ground truth of a second output of the trained neural network;

obtaining a first output from a hidden layer of the trained neural network; and

determining at least one acoustic classification of the polyphonic acoustic data based on the first output and a feature dictionary learnt from the trained neural network.

9. The system of claim 8 , wherein the determining comprises:

representing the first output as a non-negative sparse combination of the dictionary;

computing a membership function of the first output based on the non-negative sparse combination of the dictionary; and

determining an event of an acoustic classification occurred in the polyphonic acoustic data in response to a membership value of the membership function being larger than an assigned threshold.

10. The system of claim 8 , wherein the feature dictionary is learnt by:

inputting the monophonic acoustic data into the trained neural network;

obtaining a third output from a hidden layer of the trained neural network; and

learning the feature dictionary based on the third output.

11. The system of claim 10 , wherein the learning is to optimize an object function for the at least one acoustic classification.

12. The system of claim 8 , wherein the labeled monophonic acoustic data are denoted as feature vectors corresponding to a sound segment.

13. The system of claim 8 , wherein the hidden layer is the last hidden layer.

14. The system of claim 8 , wherein the trained neural network is a Recurrent Neural Network (RNN) with Long Short Term Memory (LSTM) neurons.

15. A computer program product, comprising a computer readable non-transitory storage medium having program instructions embodied therewith, the program instructions being executable by a processor to perform a method comprising:

inputting polyphonic acoustic data into a trained neural network trained by labeled monophonic acoustic data, wherein the trained neural network is trained by:

inputting the labeled monophonic acoustic data into the trained neural network; and

obtaining the trained neural network by using the label vectors of the labeled monophonic acoustic data as the ground truth of a second output of the trained neural network;

obtaining a first output from a hidden layer of the trained neural network; and

determining at least one acoustic classification of the polyphonic acoustic data based on the first output and a feature dictionary learnt from the trained neural network.

16. The computer program product of claim 15 , wherein the determining comprises:

representing the first output as a non-negative sparse combination of the dictionary;

computing a membership function of the first output based on the non-negative sparse combination of the dictionary; and

determining an event of an acoustic classification occurred in the polyphonic acoustic data in response to a membership value of the membership function being larger than an assigned threshold.

17. The computer program product of claim 15 , wherein the feature dictionary is learnt by:

inputting the monophonic acoustic data into the trained neural network;

obtaining a third output from a hidden layer of the trained neural network; and

learning the feature dictionary based on the third output.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2017
From: LIANG, XIAO XING; ZHANG, NING; ZHENG, YU LING; ZHOU, YU CHEN
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 044329/0192 →
Continuity (1)
Related Publication 20190130933A1 · May 2, 2019
Cited By (1)
US 12,300,266