IP Library Granted Patent US 10,068,446
Granted Patent B2
US 10,068,446 · App. 15/697,837 · Granted Sep 4, 2018

Detection and classification of abnormal sounds

Inventors: Ehren J. Brav (Bainbridge Island, WA); Roderick A. Hyde (Redmond, WA); Yaroslav A. Urzhumov (Bellevue, WA); Thomas A. Weaver (San Mateo, CA); Lowell L. Wood, Jr. (Bellevue, WA)
Assignee: Elwha LLC
G08B13/1672G08B3/10G08B13/19697G08B21/02G08B21/043G08B21/0469G08B29/188H04R1/028G08B17/00H04R2420/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,068,446
App. No.
15/697,837
Granted
Sep 4, 2018
Kind
B2
Abstract

An audio surveillance system includes a plurality of nodes and each node includes a microphone, a speaker, and a control unit. The microphone is configured to detect sound and the speaker is configured to provide sound. The control unit is configured to receive a plurality of inputs from the plurality of nodes and the plurality of inputs are based on a detected sound; determine a location of the source of the detected sound based on the plurality of inputs; classify the detected sound according to predefined alert conditions and based on the location of the source of the detected sound; provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and control at least one node from the plurality of nodes to provide an audio response to the detected sound.

Claims (51)

1. A system comprising:

a primary audio surveillance node including a control unit, the control unit configured to:

receive a sound input based on a sound emitted by a source and detected by a microphone configured to detect sound and acoustic pulses;

receive an acoustic pulse input based on an acoustic pulse provided by a secondary audio surveillance node and detected by the microphone;

determine a relative location of the primary audio surveillance node with respect to the secondary audio surveillance node based on the acoustic pulse;

determine a location of the source of the detected sound based on the sound input and the determined relative location; and

classify the detected sound based on a characteristic of the detected sound and the determined location of the source.

2. The system of claim 1 , wherein the primary audio surveillance node further comprises a speaker configured to emit acoustic pulses, and a wireless transceiver configured to transmit time-of-emission data for each acoustic pulse emitted by the speaker and receive time-of-emission data for each acoustic pulse provided by the secondary audio surveillance node.

3. The system of claim 1 , wherein the control unit is further configured to determine a time-of-flight for each detected acoustic pulse.

4. The system of claim 1 , wherein the primary audio surveillance node further comprises a wireless transceiver configured to transmit time-of-flight data for each detected acoustic pulse and receive time-of-emission data for an acoustic pulse detected by another audio surveillance node.

5. The system of claim 1 , wherein the determination of the relative location of the primary audio surveillance node is based on a plurality of time-of-flight data.

6. The system of claim 1 , wherein the primary audio surveillance node further comprises a wireless transceiver configured to transmit a signal based on the detected sound and receive a signal from a monitoring device regarding an audio response, wherein the control unit is configured to provide the audio response using a speaker of the primary audio surveillance node.

7. The system of claim 1 , wherein the control unit is configured to provide an alert, the alert including audio received from a monitoring device.

8. The system of claim 1 , wherein the control unit is configured to provide an alert, the alert including at least one of an image and a video.

9. The system of claim 1 , wherein the control unit is configured to determine the location of the source based on a characteristic of the detected sound, the sound characteristic at least one of a time of arrival, a relative intensity, a relative spectral content, and a triangulation.

10. The system of claim 1 , wherein the control unit is configured to automatically direct the microphone toward the determined location of the source.

11. The system of claim 1 , wherein the primary audio surveillance node further comprises a camera, and wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

12. The system of claim 11 , wherein the control unit is configured to automatically direct the camera toward the determined location of the source.

13. A method comprising:

receiving, by a control unit, a plurality of inputs from a plurality of nodes, wherein the plurality of inputs are based on a sound detected by a microphone;

determining, by the control unit, a location of a source of the detected sound based on the plurality of inputs;

classifying, by the control unit, the detected sound based on a characteristic of the detected sound and the location of the source; and

providing, by the control unit, an alert to a monitoring device regarding the detected sound based on the classification.

14. The method of claim 13 , further comprising transmitting a signal based on the detected sound and receiving a signal from the monitoring device regarding an audio response.

15. The method of claim 13 , wherein the microphone is a directional microphone.

16. The method of claim 13 , further comprising providing an audio response using a speaker of a node of the plurality of nodes, wherein the speaker includes one of a wide angle speaker, a directional speaker, or a directional speaker using nonlinearly downconverted ultrasound.

17. The method of claim 13 , further comprising receiving, by the control unit, an image captured by a camera, wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

18. The method of claim 17 , further comprising automatically directing the microphone based on the image.

19. The method of claim 13 , wherein the classification is based on a predefined alert condition.

20. The method of claim 19 , further comprising altering the predefined alert condition based on a time of day.

21. The method of claim 19 , wherein the predefined alert condition comprises a first alert condition and a second alert condition, and wherein the first alert condition is applied to a first node and a second alert condition is applied to a second node.

22. The method of claim 21 , wherein the applied alert condition is based on the location of the node.

23. The method of claim 13 , further comprising determining the location of at least a first node of the plurality of nodes based on an acoustic pulse transmitted by a speaker of a second node and received by a microphone of at least one of the first node and a third node.

24. The method of claim 13 , further comprising detecting, by the control unit, a moving audio shadow caused by a portion of the source being blocked.

25. A method for detecting and classifying sounds, comprising:

receiving, by a control unit, sound data from a plurality of audio surveillance nodes, the sound data indicative of a sound detected by at least one of the audio surveillance nodes and an acoustic pulse received by a primary audio surveillance node and transmitted by a secondary audio surveillance node;

determining, by the control unit, a time-of-flight of the acoustic pulse;

determining, by the control unit, a location of the primary audio surveillance node based on the time-of-flight;

determining, by the control unit, a location of a source of the detected sound based on the sound data and based on the location of the primary audio surveillance node;

classifying, by the control unit, the detected sound based on the location of the source; and

controlling, by the control unit, a speaker to provide an audio response to the detected sound.

26. The method of claim 25 , wherein classifying the detected sound is based on a predefined alert condition, the predefined alert condition including a sound characteristic.

27. The method of claim 26 , further comprising altering the predefined alert condition based on a time of day.

28. The method of claim 26 , wherein the predefined alert condition includes a condition based on at least one of a voice, glass breaking, running, falling, a scream, fighting, or a gun shot.

29. The method of claim 26 , wherein the predefined alert condition is updated based on the detected sound.

30. The method of claim 26 , wherein the predefined alert condition is updated based on a user input received at a monitoring device.

31. The method of claim 25 , further comprising determining a movement of the source based on Doppler shifts in the detected sound.

32. The method of claim 25 , further comprising determining a velocity of the source based on the sound data.

33. The method of claim 25 , wherein the detected sound is detected by a microphone, and wherein the microphone is configured to detect sound only during a predefined time period.

34. The method of claim 33 , wherein the predefined time period occurs once per second.

35. The method of claim 25 , further comprising providing an audio response to the detected sound based on at least one of the classification of the detected sound or a user input received at a monitoring device.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2022
From: JUROSENSE, LLC
To: THE INVENTION SCIENCE FUND II, LLC
Reel/Frame 059357/0336 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2021
From: ELWHA LLC
To: THE INVENTION SCIENCE FUND II, LLC
Reel/Frame 057633/0138 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2021
From: THE INVENTION SCIENCE FUND II, LLC
To: JUROSENSE, LLC
Reel/Frame 057633/0502 →
Continuity (3)
Continuation 15209130 · Jul 13, 2016
Continuation 14562282 · Dec 5, 2014
Related Publication 20170372571A1 · Dec 28, 2017