IP Library Granted Patent US 10,978,050
Granted Patent B2
US 10,978,050 · App. 16/280,806 · Granted Apr 13, 2021

Audio type detection

Inventors: Krishna Khadloya (San Jose, CA); Chandan Gope (Cupertino, CA); Vaidhi Nathan (San Jose, CA)
Assignee: INTELLIVISION TECHNOLOGIES CORP.
G10L15/16G06F9/542G06N3/0454G06N3/08G06N20/10G06N20/20G10L25/18G10L25/21
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,978,050
App. No.
16/280,806
Granted
Apr 13, 2021
Kind
B2
Abstract

Artificial intelligence-based processing can be used to classify audio information received from an audio input unit. In an example, audio information can be received from a microphone configured to monitor an environment. A processor circuit can identify identifying one or more features of the audio information received from the microphone and use a first applied machine learning algorithm to analyze the one or more features and determine whether the audio information includes an indication of an abnormal event in the environment. In an example, the processor circuit can use a different second applied machine learning algorithm, such as a neural network-based deep learning algorithm, to analyze the same one or more features and classify the audio information as including an indication of a particular event type in the environment.

Claims (39)

1. A method for using artificial intelligence-based processing to classify audio information received from an audio input unit, the method comprising:

receiving audio information from a microphone configured to monitor an environment; and

using a processor circuit:

identifying one or more features of the audio information received from the microphone;

using a first applied machine learning algorithm to analyze the one or more features, determining whether the audio information includes an indication of an abnormal event in the environment;

using a different second applied machine learning algorithm to analyze the same one or more features, classifying the audio information as including an indication of a particular event type in the environment; and

generating an alert about the particular event type.

2. The method of claim 1 , wherein the classifying the audio information includes confirming or refuting the indication of an abnormal event in the environment.

3. The method of claim 1 , wherein using the first applied machine learning algorithm includes using the processor circuit to use support vector machines or a neural network to determine whether the audio information includes the indication of the abnormal event.

4. The method of claim 1 , wherein the first applied machine learning algorithm includes a neural network-based deep learning algorithm.

5. The method of claim 1 , wherein the using the first or different second applied machine learning algorithm includes using the processor circuit embedded in a smart speaker device.

6. The method of claim 1 , wherein using the different second applied machine learning algorithm includes using the processor circuit to use a deep learning neural network-based algorithm to classify the audio information.

7. The method of claim 1 , wherein the classifying the audio information includes identifying whether the audio information includes an acoustic signature of one or more of breaking glass, a gun shot, a dog bark, a security alarm, a fire alarm, a smoke alarm, a water alarm, human voices, or human crying.

8. The method of claim 1 , further comprising determining a loudness characteristic of the received audio information and wherein the identifying the one or more features of the audio information is conditioned on the loudness characteristic exceeding a specified minimum loudness threshold.

9. The method of claim 1 , wherein the identifying the one or more features of the audio information includes determining a power spectrum or spectrogram, and wherein the determining whether the audio information includes the indication of the abnormal event includes using the power spectrum or spectrogram.

10. The method of claim 1 , wherein the generating the alert includes communicating an alert to a user who is associated with a smart speaker, wherein the smart speaker includes the microphone.

11. The method of claim 1 , further comprising training the first and/or different second applied machine learning algorithm using hard negatives by selecting, as training data, results for which the algorithm provides a false alarm.

12. The method of claim 1 , further comprising identifying a multi-dimensional spectrogram based on the audio information from the microphone; and

wherein using the different second applied machine learning algorithm includes using the different second applied machine learning algorithm to analyze the multi-dimensional spectrogram to classify the audio information as including an indication of the particular event type in the environment.

13. A system for performing automated audible event detection and classification, the system comprising:

an audio sensor configured to monitor an environment; and

a processor circuit configured to:

identify one or more features of the audio information received from the microphone;

use a first applied machine learning algorithm to analyze the one or more features and determine whether the audio information includes an indication of an abnormal event in the environment;

use a different second applied machine learning algorithm to analyze the same one or more features and classify the audio information as including an indication of a particular event type in the environment; and

communicate an alert about the particular event type to a user of the system.

14. The system of claim 13 , further comprising a memory circuit that includes a reference data set for use by the first or different second applied machine learning algorithms, wherein the reference data set includes positive target samples and hard negatives, wherein the hard negatives comprise training data that is based on false alarms.

15. The system of claim 13 , wherein the audio sensor and the processor circuit are embedded in a smart speaker or camera device.

16. The system of claim 13 , wherein the processor circuit is configured to use the different second applied machine learning algorithm to classify the audio information as including an acoustic signature of one or more of breaking glass, a gun shot, a dog bark, a security alarm, a fire alarm, a smoke alarm, a water alarm, human voices, or human crying.

17. The system of claim 13 , wherein the processor circuit is configured to determine a loudness characteristic of the received audio information and wherein the processor is configured to identify the one or more features of the audio information only when the loudness characteristic exceeds a specified minimum loudness threshold.

18. The system of claim 13 , wherein the processor circuit is configured to identify a multi-dimensional spectrogram of the audio information, and wherein the one or more features of the audio information includes the spectrogram.

19. A smart speaker for monitoring activities in an environment, the smart speaker comprising:

an audio receiver circuit configured to receive acoustic information from a microphone in an environment and generate a series of overlapping audio sample frames representative of the acoustic information; and

a processor circuit configured to:

identify a power spectrum of the acoustic information received from the microphone when the power spectrum indicates that the acoustic information includes an audible event and the audible event has greater than a specified threshold loudness characteristic;

use a first applied machine learning algorithm to analyze the power spectrum of the acoustic information and determine whether the acoustic information includes an indication of an abnormal event in the environment;

use a neural network-based deep learning algorithm to analyze the same power spectrum of the acoustic information and classify the acoustic information as including an indication of a particular event type in the environment; and

communicate an alert about the particular event type to a user of the system.

20. The smart speaker of claim 19 , wherein the processor circuit is configured to classify the acoustic information as including an acoustic signature of one or more of breaking glass, a gun shot, a dog bark, a security alarm, a fire alarm, a smoke alarm, a water alarm, human voices, or human crying, and wherein the processor circuit is configured to communicate to the user a portion of the acoustic information that corresponds to the acoustic signature.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2024
From: INTELLIVISION TECHNOLOGIES CORP.
To: NICE NORTH AMERICA LLC
Reel/Frame 068815/0671 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2020
From: KHADLOYA, KRISHNA; GOPE, CHANDAN; NATHAN, VAIDHI
To: INTELLIVISION TECHNOLOGIES CORP.
Reel/Frame 052322/0975 →
Continuity (2)
Provisional Application 62632421 · Feb 20, 2018
Related Publication 20190259378A1 · Aug 22, 2019
Cited By (4)
US 12,361,807 US 12,373,540 US 12,382,001 US 12,694,774