IP Library Granted Patent US 10,585,941
Granted Patent B2
US 10,585,941 · App. 14/447,227 · Granted Mar 10, 2020

Audio object search and analysis system

Inventor: Douglas C. Garrett (Santa Monica, CA)
Assignee: ACE METRIX, INC.
G06F16/683
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,585,941
App. No.
14/447,227
Granted
Mar 10, 2020
Kind
B2
Abstract

An audio file is transformed into a Gabor spectrogram. This is used to compare the audio file to a database of audio files, each represented as a Gabor spectrogram. Before two spectrograms are compared, they are aligned. The spectrograms are broken into blocks and individual Gabor vectors in the blocks are compared. Similarities are stored and an aggregate similarity value is derived for the block. After a series of such comparisons and shifting of the secondary spectrogram block, essentially a running window, an offset value is determined. This offset is used to align the two spectrograms at which stage the spectrograms can be compared in a more effective and meaningful manner. A set of observables is derived from the comparisons and the primary spectrogram is classified in way suitable for the application environment.

Claims (44)

1. A method of comparing sample audio files with each of a plurality of audio files stored in a database system, and classifying the plurality of audio files based on the comparing, the method comprising:

receiving, via a network interface, a first audio signal;

creating, using a processor, a first Gabor representation of the first audio signal of the plurality of audio files stored in a storage location of at least one storage device of the database system;

aligning, using the processor, the first Gabor representation with an existing second Gabor representation of a second audio signal of the plurality of audio files, the aligning being based on a comparison between a plurality of Gabor vectors of the first Gabor representation with a plurality of Gabor vectors included in the second Gabor representation;

executing, using the processor, a similarity scan between the first Gabor representation and the second Gabor representation, the similarity scan using a running window to compare a chain of Gabor vectors included in a first block of the first Gabor representation with corresponding chains of Gabor vectors included in blocks of the second Gabor representation, the corresponding chains of Gabor vectors being identified and compared based, at least in part, on a correspondence map generated during the aligning;

obtaining, using the processor, similarity-related data resulting from the similarity scan, the similarity-related data being stored in a memory device coupled to the processor;

analyzing, using the processor, similarity-related data to determine degrees of similarity between the first audio signal and the second audio signal;

storing a representation of the degrees of similarity at the storage location of the at least one storage device associated with the plurality of audio files; and

classifying at least the first audio signal based, at least in part, on the degrees of similarity.

2. A method as recited in claim 1 wherein creating a first Gabor representation of a first audio signal further comprises:

executing a Gabor transformation on the first audio signal thereby producing a plurality of audio signals;

creating a plurality of Gabor vectors from the plurality of audio signals; and

determining similarity between each of the vectors in the plurality of Gabor vectors and Gabor vectors from a second audio signal.

3. A method as recited in claim 1 wherein aligning the first Gabor representation further comprises:

creating multiple blocks from a first Gabor spectrogram, each block having multiple Gabor vectors,

wherein the running window is from a second Gabor spectrogram.

4. A method as recited in claim 3 further comprising:

deriving a first similarity from the comparison; and

aggregating the first similarity with a plurality of similarity values thereby creating an aggregated similarity for the block.

5. A method as recited in claim 1 wherein analyzing similarity-related data further comprises:

classifying the first audio signal as duplicate, near duplicate, and not duplicate with respect to the second audio signal.

6. A method as recited in claim 1 further comprising:

breaking the first audio signal into multiple blocks, a block having a plurality of Gabor vectors.

7. A method as recited in claim 1 further comprising:

determining where a block has the highest similarity in the second audio signal at a specific time; and

storing a similarity value representing this highest similarity.

8. A method as recited in claim 1 further comprising:

determining an offset between two audio signals based on where the highest number of same similarities are, wherein similarity may be low but if multiple blocks have the same offset, then using the offset for alignment.

9. A method as recited in claim 1 wherein similarity-related data includes average similarity, maximum similarity, minimum similarity, similarity standard deviation, entropy related to aggregate similarity value.

10. A method as recited in claim 1 further comprising:

storing the aggregate similarity value; and creating a histogram of similarity values.

11. A method of aligning a plurality of sample audio files and a plurality of audio files stored in a database system, and classifying the plurality of audio files, the method comprising:

receiving, via a network interface, a first audio file and a second audio file;

creating, using a processor, a first Gabor spectrogram of the first audio file and a second Gabor spectrogram of the second audio file;

breaking, using the processor, the first spectrogram into a plurality of blocks, each block including a chain of Gabor vectors;

creating, using the processor, a running window of Gabor vectors for the second spectrogram, the number of Gabor vectors being the same as the number of Gabor vectors in each block;

comparing, using the processor, a Gabor vector from the chain of Gabor vectors of a first block of the first Gabor spectrogram to a corresponding Gabor vector included in a second block of the running window associated with the second Gabor spectrogram, the corresponding chains of Gabor vectors being identified and compared based, at least in part, on a correspondence map;

storing, using the processor, similarities from comparisons in a memory device coupled with the processor;

creating, using the processor, a plurality of aggregate similarities from the similarities;

deriving, using the processor, a plurality of offsets from the plurality of aggregate similarities;

determining, using the processor, a best offset based on a peak of a histogram of the plurality of offsets; and

classifying at least the first audio file based, at least in part, on the similarities.

12. A method as recited in claim 11 further comprising:

utilizing the best offset to align the first spectrogram and the second spectrogram.

Assignments (6)
SECURITY INTEREST Recorded Mar 8, 2026
From: ISPOT.TV, INC.; 605, LLC
To: HERCULES CAPITAL, INC., AS AGENT
Reel/Frame 074006/0123 →
RELEASE OF SECURITY INTEREST Recorded May 1, 2025
From: FIRST-CITIZENS BANK & TRUST COMPANY
To: ISPOT.TV INC.; 605, LLC
Reel/Frame 071002/0711 →
SECURITY INTEREST Recorded Feb 1, 2025
From: ISPOT.TV, INC.
To: HERCULES CAPITAL, INC., AS AGENT
Reel/Frame 070082/0461 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 18, 2022
From: ACE METRIX, INC.
To: ISPOT.TV, INC.
Reel/Frame 059052/0123 →
SECURITY INTEREST Recorded Dec 31, 2020
From: ACE METRIX, INC.
To: SILICON VALLEY BANK
Reel/Frame 054787/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2014
From: GARRETT, DOUGLAS C.
To: ACE METRIX, INC.
Reel/Frame 033713/0379 →