IP Library Granted Patent US 9,749,762
Granted Patent B2
US 9,749,762 · App. 15/209,251 · Granted Aug 29, 2017

Facilitating inferential sound recognition based on patterns of sound primitives

Inventors: Sebastien J. V. Christian (Mountain View, CA); Thor C. Whalen (Menlo Park, CA)
Assignee: OtoSense, Inc.
H04R29/00G08B17/10G08B21/0423G08B21/18G08B21/182G10L21/14G10L25/27G10H2210/301
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,749,762
App. No.
15/209,251
Granted
Aug 29, 2017
Kind
B2
Abstract

The disclosed embodiments provide a system that performs a sound-recognition operation. During operation, the system recognizes a sequence of sound primitives in an audio stream, wherein a sound primitive is associated with a semantic label comprising one or more words that describe a sound characterized by the sound primitive. Next, the system feeds the sequence of sound primitives into a finite-state automaton that recognizes events associated with sequences of sound primitives. Finally, the system feeds the recognized events into an output system that generates an output associated with the recognized events to be displayed to a user.

Claims (52)

1. A method for performing a sound-recognition operation, comprising:

recognizing a sequence of sound primitives in an audio stream, wherein a sound primitive is associated with a semantic label comprising one or more words that describe a sound characterized by the sound primitive, wherein recognizing the sequence of sound primitives comprises,

performing a feature-detection operation on a sequence of sound samples from the audio stream to detect a set of sound features, wherein each sound feature comprises a measurable characteristic for a time window of consecutive sound samples, and wherein detecting the sound feature involves generating a coefficient indicating a likelihood that the sound feature is present in the time window,

creating a set of feature vectors from coefficients generated by the feature-detection operation, wherein each feature vector comprises a set of coefficients for sound features in the set of sound features, and

identifying the sequence of sound primitives from the sequence of feature vectors;

feeding the sequence of sound primitives into a finite-state automaton that recognizes events associated with sequences of sound primitives; and

feeding the recognized events into an output system that generates an output associated with the recognized events to be displayed to a user.

2. The method of claim 1 ,

wherein the finite-state automaton is a non-deterministic finite-state automaton that can exist in multiple states at the same time; and

wherein the non-deterministic finite-state automaton maintains a probability value for each of the multiple states that the finite-state automaton can exist in.

3. The method of claim 1 , wherein feeding the sequence of sound primitives into the finite-state automaton comprises:

feeding the sequence of sound primitives into a first-level finite-state automaton that recognizes first-level events from the sequence of sound primitives to generate a sequence of first-level events;

feeding the sequence of first-level events into a second-level finite-state automaton that recognizes second-level events from the sequence of first-level events to generate a sequence of second-level events; and

repeating the process for zero or more additional levels of finite-state automatons to generate the recognized events.

4. The method of claim 3 , wherein if a probability value for a state in the non-deterministic finite-state automaton does not meet an activation-potential-related threshold value after a state-transition operation, the probability value for the state is set to zero.

5. The method of claim 3 , wherein the finite-state automaton performs state-transition operations by performing computations involving one or more sequence matrices containing coefficients that define state transitions.

6. The method of claim 1 , wherein the output system triggers an alert when a probability that a tracked event is occurring exceeds a threshold value.

7. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a sound-recognition operation, the method comprising:

recognizing a sequence of sound primitives in an audio stream, wherein a sound primitive is associated with a semantic label comprising one or more words that describe a sound characterized by the sound primitive, wherein recognizing the sequence of sound primitives comprises,

performing a feature-detection operation on a sequence of sound samples from the audio stream to detect a set of sound features, wherein each sound feature comprises a measurable characteristic for a time window of consecutive sound samples, and wherein detecting the sound feature involves generating a coefficient indicating a likelihood that the sound feature is present in the time window,

creating a set of feature vectors from coefficients generated by the feature-detection operation, wherein each feature vector comprises a set of coefficients for sound features in the set of sound features, and

identifying the sequence of sound primitives from the sequence of feature vectors;

feeding the sequence of sound primitives into a finite-state automaton that recognizes events associated with sequences of sound primitives; and

feeding the recognized events into an output system that generates an output associated with the recognized events to be displayed to a user.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the finite-state automaton is a non-deterministic finite-state automaton that can exist in multiple states at the same time; and

wherein the non-deterministic finite-state automaton maintains a probability value for each of the multiple states that the finite-state automaton can exist in.

9. The non-transitory computer-readable storage medium of claim 7 , wherein feeding the sequence of sound primitives into the finite-state automaton comprises:

feeding the sequence of sound primitives into a first-level finite-state automaton that recognizes first-level events from the sequence of sound primitives to generate a sequence of first-level events;

feeding the sequence of first-level events into a second-level finite-state automaton that recognizes second-level events from the sequence of first-level events to generate a sequence of second-level events; and

repeating the process for zero or more additional levels of finite-state automatons to generate the recognized events.

10. The non-transitory computer-readable storage medium of claim 9 , wherein if a probability value for a state in the non-deterministic finite-state automaton does not meet an activation-potential-related threshold value after a state-transition operation, the probability value for the state is set to zero.

11. The non-transitory computer-readable storage medium of claim 9 , wherein the finite-state automaton performs state-transition operations by performing computations involving one or more sequence matrices containing coefficients that define state transitions.

12. The non-transitory computer-readable storage medium of claim 7 , wherein the output system triggers an alert when a probability that a tracked event is occurring exceeds a threshold value.

13. A system that performs a sound-recognition operation, comprising:

at least one processor and at least one associated memory; and

a sound-recognition system that executes on the at least one processor, wherein during operation, the sound-recognition system,

recognizes a sequence of sound primitives in an audio stream, wherein a sound primitive is associated with a semantic label comprising one or more words that describe a sound characterized by the sound primitive, wherein while recognizing the sequence of sound primitives, the sound-recognition system,

performs a feature-detection operation on a sequence of sound samples from the audio stream to detect a set of sound features, wherein each sound feature comprises a measurable characteristic for a time window of consecutive sound samples, and wherein detecting the sound feature involves generating a coefficient indicating a likelihood that the sound feature is present in the time window,

creates a set of feature vectors from coefficients generated by the feature-detection operation, wherein each feature vector comprises a set of coefficients for sound features in the set of sound features, and

identifies the sequence of sound primitives from the sequence of feature vectors;

feeds the sequence of sound primitives into a finite-state automaton that recognizes events associated with sequences of sound primitives, and

feeds the recognized events into an output system that generates an output associated with the recognized events to be displayed to a user.

14. The system of claim 13 ,

wherein the finite-state automaton is a non-deterministic finite-state automaton that can exist in multiple states at the same time; and

wherein the non-deterministic finite-state automaton maintains a probability value for each of the multiple states that the finite-state automaton can exist in.

15. The system of claim 14 , wherein if a probability value for a state in the non-deterministic finite-state automaton does not meet an activation-potential-related threshold value after a state-transition operation, the probability value for the state is set to zero.

16. The system of claim 15 , wherein the finite-state automaton performs state-transition operations by performing computations involving one or more sequence matrices containing coefficients that define state transitions.

17. The system of claim 13 , wherein while feeding the sequence of sound primitives into the finite-state automaton, the sound-recognition system:

feeds the sequence of sound primitives into a first-level finite-state automaton that recognizes first-level events from the sequence of sound primitives to generate a sequence of first-level events;

feeds the sequence of first-level events into a second-level finite-state automaton that recognizes second-level events from the sequence of first-level events to generate a sequence of second-level events; and

repeats the process for zero or more additional levels of finite-state automatons to generate the recognized events.

18. The system of claim 13 , wherein the output system triggers an alert when a probability that a tracked event is occurring exceeds a threshold value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: OTOSENSE, INC.
To: ANALOG DEVICES, INC.
Reel/Frame 053098/0719 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2017
From: CHRISTIAN, SEBASTIEN J.V.; WHALEN, THOR C.
To: OTOSENSE, INC.
Reel/Frame 042141/0481 →
Continuity (4)
Continuation In Part 14616627 · Feb 6, 2015
Provisional Application 61936706 · Feb 6, 2014
Provisional Application 62387126 · Dec 23, 2015
Related Publication 20160330557A1 · Nov 10, 2016