Granted Patent B2

US 9,111,547 · App. 13/591,489 · Granted Aug 18, 2015

Inventors: Alexander C. Loui (Penfield, NY); Wei Jiang (Fairport, NY); Kevin Michael Gobeyn (Rochester, NY); Charles Parker (Corvallis, OR)

Assignee: KODAK ALARIS INC.

G10L25/51

Abstract

Claims (48)

1. A method for determining a semantic concept associated with an audio signal captured using an audio sensor, comprising:

receiving the audio signal from the audio sensor;

using a data processor to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, the semantic concept detectors being associated with a corresponding plurality of semantic concepts, each semantic concept detector being adapted to detect a particular semantic concept;

using a data processor to automatically analyze the preliminary semantic concept detection values using a joint likelihood model to determine updated semantic concept detection values; wherein the joint likelihood model determines the updated semantic concept detection values based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur;

identifying one or more semantic concept associated with the audio signal based on the updated semantic concept detection values; and

storing an indication of the identified semantic concepts in a processor-accessible memory;

wherein the semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts, and

wherein each of the semantic concept detectors determines the preliminary semantic concept detection values responsive to an associated set of audio features, the audio features being determined by analyzing the audio signal.

2. The method of claim 1 wherein the particular audio features associated with each semantic concept detector are determined during the joint training process.

3. The method of claim 1 wherein the audio signal is subdivided into a set of audio frames, and wherein the audio frames are analyzed to determine frame-level audio features.

4. The method of claim 3 wherein the frame-level audio features from a plurality of audio frames are aggregated to determine clip-level features.

5. The method of claim 4 wherein the frame-level audio features are aggregated by computing frame-level preliminary semantic concept detection values responsive to the frame-level audio features and then determining clip-level preliminary semantic concept detection values by determining an average or a maximum of the frame-level preliminary semantic concept detection values.

6. The method of claim 1 wherein the semantic concept detectors are Nearest Neighbor classifiers, Support Vector Machine classifiers or decision tree classifiers.

7. The method of claim 1 wherein the joint likelihood model is a Markov Random Field model having a set of nodes connected by edges, wherein each node corresponds to a particular semantic concept, and the edge connecting a pair of nodes corresponds to a pair-wise potential function between the corresponding pair of semantic concepts providing an indication of the pair-wise likelihood that the pair of semantic concepts co-occur.

8. The method of claim 1 further including applying a filtering process to discard any semantic concept having a preliminary semantic concept detection value below a predefined threshold.

9. The method of claim 1 wherein the joint training process determines the semantic concept detectors and the joint likelihood model that maximize a predefined performance assessment function.

10. A method for determining a semantic concept associated with an audio signal captured using an audio sensor, comprising:

receiving the audio signal from the audio sensor;

using a data processor to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, the semantic concept detectors being associated with a corresponding plurality of semantic concepts, each semantic concept detector being adapted to detect a particular semantic concept;

using a data processor to automatically analyze the preliminary semantic concept detection values using a joint likelihood model to determine updated semantic concept detection values; wherein the joint likelihood model determines the updated semantic concept detection values based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur;

identifying one or more semantic concept associated with the audio signal based on the updated semantic concept detection values; and

storing an indication of the identified semantic concepts in a processor-accessible memory;

wherein the semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts, and

wherein the semantic concept detectors are Nearest Neighbor classifiers, Support Vector Machine classifiers or decision tree classifiers.

11. A method for determining a semantic concept associated with an audio signal captured using an audio sensor, comprising:

receiving the audio signal from the audio sensor;

using a data processor to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, the semantic concept detectors being associated with a corresponding plurality of semantic concepts, each semantic concept detector being adapted to detect a particular semantic concept;

using a data processor to automatically analyze the preliminary semantic concept detection values using a joint likelihood model to determine updated semantic concept detection values; wherein the joint likelihood model determines the updated semantic concept detection values based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur;

identifying one or more semantic concept associated with the audio signal based on the updated semantic concept detection values; and

storing an indication of the identified semantic concepts in a processor-accessible memory;

wherein the semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts, and

wherein the joint likelihood model is a Markov Random Field model having a set of nodes connected by edges, wherein each node corresponds to a particular semantic concept, and the edge connecting a pair of nodes corresponds to a pair-wise potential function between the corresponding pair of semantic concepts providing an indication of the pair-wise likelihood that the pair of semantic concepts co-occur.

12. A method for determining a semantic concept associated with an audio signal captured using an audio sensor, comprising:

receiving the audio signal from the audio sensor;

using a data processor to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, the semantic concept detectors being associated with a corresponding plurality of semantic concepts, each semantic concept detector being adapted to detect a particular semantic concept;

using a data processor to automatically analyze the preliminary semantic concept detection values using a joint likelihood model to determine updated semantic concept detection values; wherein the joint likelihood model determines the updated semantic concept detection values based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur;

identifying one or more semantic concept associated with the audio signal based on the updated semantic concept detection values;

storing an indication of the identified semantic concepts in a processor-accessible memory; and

applying a filtering process to discard any semantic concept having a preliminary semantic concept detection value below a predefined threshold;

wherein the semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts.

13. A method for determining a semantic concept associated with an audio signal captured using an audio sensor, comprising:

receiving the audio signal from the audio sensor;

using a data processor to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, the semantic concept detectors being associated with a corresponding plurality of semantic concepts, each semantic concept detector being adapted to detect a particular semantic concept;

using a data processor to automatically analyze the preliminary semantic concept detection values using a joint likelihood model to determine updated semantic concept detection values; wherein the joint likelihood model determines the updated semantic concept detection values based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur;

identifying one or more semantic concept associated with the audio signal based on the updated semantic concept detection values; and

storing an indication of the identified semantic concepts in a processor-accessible memory;

wherein the semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts, and

wherein the joint training process determines the semantic concept detectors and the joint likelihood model that maximize a predefined performance assessment function.

Assignments (12)

Continuity (1)