Granted Patent B2

US 12,586,592 · App. 17/882,447 · Granted Mar 24, 2026

Inventors: Shrirang Jangi (Acton, MA); Vilas Bhade (Acton, MA)

Assignee: Ribbon Communications Operating Company, Inc.

G10L19/02H03M7/3073

Abstract

Claims (113)

1 . A method of processing an audio signal comprising:

removing silence from the audio signal, said audio signal being from a first call;

after the silence has been removed from the audio signal, determining, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins, N being an integer greater than 1;

identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks;

generating a first audio fingerprint from at least some of the identified plurality of dominant frequency peaks and the identified positions in the audio signal corresponding to the identified peaks, said first audio fingerprint being an ordered set of information including: a first frequency bin value, a second frequency bin value, and a delta time, said first frequency bin value corresponding to a first dominant frequency peak, said first dominant frequency peak being one of the identified dominant frequency peaks, said second frequency bin value corresponding to a second dominant frequency peak, said second dominant frequency peak being one of the identified dominant frequency peaks, said first dominant frequency peak and said second dominant frequency peak being different dominant frequency peaks, said delta time being a time difference between a second identified location in the audio signal corresponding to the second dominant frequency peak and the first identified location in the audio signal corresponding to the first dominant frequency peak;

generating a first set of fuzzy audio fingerprints from the first audio fingerprint, said generating the first set of fuzzy audio fingerprints from the first audio fingerprint including modifying one or more of: the first frequency bin value or the second frequency bin value, said first frequency bin value identifying a first range of frequencies corresponding to a first frequency bin, said second frequency bin value identifying a second range of frequencies corresponding to a second frequency bin; and

using the first audio fingerprint and the first set of fuzzy audio fingerprints to determine whether the first call is a robocall; and

wherein said modifying includes performing one of the following:

(i) changing the first frequency bin value to a third frequency bin value, said third frequency bin value identifying a third range of frequencies corresponding to a third frequency bin, said third range of frequencies being different than said first range of frequencies,

(ii) changing the second frequency bin value to a fourth frequency bin value, said fourth frequency bin value identifying a fourth range of frequencies corresponding to a fourth frequency bin, said fourth range of frequencies being different than said second range of frequencies, or

(iii) changing the first frequency bin value to the third frequency bin value, said third frequency bin value identifying the third range of frequencies corresponding to the third frequency bin, said third range of frequencies being different than said first range of frequencies and changing the second frequency bin value to the fourth frequency bin value, said fourth frequency bin value identifying the fourth range of frequencies corresponding to the fourth frequency bin, said fourth range of frequencies being different than said second range of frequencies; and

wherein said plurality of N different frequency bins includes said first frequency bin, said second frequency bin, said third frequency bin, and said fourth frequency bin.

2 . The method of claim 1 ,

wherein said identifying a plurality of dominant frequency peaks based on the determined power spectral density values includes: identifying for each of the plurality of time segments of the audio signal a set of frequency bins with the highest power spectral density values above a first threshold value, said set of frequency bins having M or fewer entries, where M is less than N, and where M is an integer; and

wherein said identified positions in the audio signal corresponding to the identified peaks are times corresponding to the time segments in which the identified peaks appear.

3 . The method of claim 2 ,

wherein said first audio fingerprint further includes a first time;

wherein said first time is a first identified location in the audio signal corresponding to the first dominant frequency peak, said first time being a time corresponding to a first time segment of the plurality of time segments, said first dominant frequency peak appearing in said first time segment.

4 . The method of claim 2 , further comprising:

generating a first fingerprint-set for the first call, said generating a first fingerprint-set for the first call including generating a plurality of audio fingerprints from the identified plurality of dominant frequency peaks and the identified positions in the audio signal corresponding to the identified peaks, said first audio fingerprint being one of said plurality of audio fingerprints.

5 . The method of claim 4 , further comprising:

generating a fingerprint-set dictionary for the first call without using a hash function for audio fingerprints in the fingerprint-set dictionary, said fingerprint-set dictionary including a key value identifying individual fingerprints for the first call, and a list of time entries identifying individual fingerprints in the fingerprint-set for the call by the time in the audio signal to which the individual fingerprint corresponds.

6 . The method of claim 1 , further comprising:

performing, prior to said identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks, a filtering operation on the audio signal to remove high frequency signals above a first frequency threshold level.

7 . The method of claim 1 , further comprising:

quantizing the determined power spectral density (PSD) values of the audio signal.

8 . The method of claim 7 ,

wherein said step of identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks includes:

generating a spectrogram of power spectral density values based on: (i) said determined power spectral density values of the audio signal, (ii) the plurality of N different frequency bins, and (iii) the plurality of time segments; and

applying a maximal filter to said spectrogram of power spectral density values to locate frequency peaks in said spectrogram.

9 . The method of claim 8 ,

wherein said step of identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks further includes:

applying an erosion filter to said spectrogram of power spectral density values after applying said maximal filter.

10 . The method of claim 1 ,

wherein said audio signal is digitally encoded audio; and

wherein said method of processing said audio signal further includes prior to determining, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins:

decoding said digitally encoded audio; and

converting a sampling rate for said audio to an 8 KHz sampling rate when said sampling rate is not 8 KHz.

11 . The method of claim 1 ,

wherein said step of removing silence from the audio signal includes: (i) using voice activation detection to determine portions of the audio signal with a signal level less than a first threshold value, said portions of the audio signal being less than the first threshold value being determined to be silence; and (ii) removing portions of the audio signal determined to be silence; and

wherein said step of removing silence from the audio signal is performed using a voice activated detector.

12 . The method of claim 1 , further compromising:

receiving, by a Session Border Controller, the audio signal of the first call;

wherein said first audio fingerprint is generated in real-time by the Session Border Controller as said audio signal passes through said Session Border Controller.

13 . The method of claim 1 , wherein said generating the first set of fuzzy audio fingerprints from the first audio fingerprint further includes:

modifying the delta time of the first audio fingerprint.

14 . The method of claim 1 , wherein said generating the first set of fuzzy audio fingerprints from the first audio fingerprint including modifying one or more of: the first frequency bin value or the second frequency bin value includes:

modifying the first frequency bin value in a logarithmic manner based on the first frequency range to which the first frequency bin value corresponds.

15 . A system for processing an audio signal comprising:

an audio fingerprinting device including a first processor, said first processor controlling the audio fingerprinting device to perform the following operations:

removing silence from the audio signal, said audio signal being from a first call;

after the silence has been removed from the audio signal, determining, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins, N being an integer greater than 1;

identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks;

generating a first audio fingerprint from at least some of the identified plurality of dominant frequency peaks and the identified positions in the audio signal corresponding to the identified peaks, said first audio fingerprint being an ordered set of information including: a first frequency bin value, a second frequency bin value, and a delta time, said first frequency bin value corresponding to a first dominant frequency peak, said first dominant frequency peak being one of the identified dominant frequency peaks, said second frequency bin value corresponding to a second dominant frequency peak, said second dominant frequency peak being one of the identified dominant frequency peaks, said first dominant frequency peak and said second dominant frequency peak being different dominant frequency peaks, said delta time being a time difference between a second identified location in the audio signal corresponding to the second dominant frequency peak and the first identified location in the audio signal corresponding to the first dominant frequency peak;

generating a first set of fuzzy audio fingerprints from the first audio fingerprint by modifying one or more of: the first frequency bin value or the second frequency bin value, said first frequency bin value identifying a first range of frequencies corresponding to a first frequency bin, said second frequency bin value identifying a second range of frequencies corresponding to a second frequency bin; and

using the first audio fingerprint and the first set of fuzzy audio fingerprints to determine whether the first call is a robocall; and

wherein said modifying includes performing one of the following:

(i) changing the first frequency bin value to a third frequency bin value, said third frequency bin value identifying a third range of frequencies corresponding to a third frequency bin, said third range of frequencies being different than said first range of frequencies,

(ii) changing the second frequency bin value to a fourth frequency bin value, said fourth frequency bin value identifying a fourth range of frequencies corresponding to a fourth frequency bin, said fourth range of frequencies being different than said second range of frequencies, or

(iii) changing the first frequency bin value to the third frequency bin value, said third frequency bin value identifying the third range of frequencies corresponding to the third frequency bin, said third range of frequencies being different than said first range of frequencies and changing the second frequency bin value to the fourth frequency bin value, said fourth frequency bin value identifying the fourth range of frequencies corresponding to the fourth frequency bin, said fourth range of frequencies being different than said second range of frequencies; and

wherein said plurality of N different frequency bins includes said first frequency bin, said second frequency bin, said third frequency bin, and said fourth frequency bin.

16 . The system of claim 15 ,

wherein said identifying a plurality of dominant frequency peaks based on the determined power spectral density values includes: identifying for each of the plurality of time segments of the audio signal a set of frequency bins with the highest power spectral density values above a first threshold value, said set of frequency bins having M or fewer entries, where M is less than N, and where M is an integer; and

wherein said identified positions in the audio signal corresponding to the identified peaks are times corresponding to the time segments in which the identified peaks appear.

17 . The system of claim 15 ,

wherein said first processor further controls the audio fingerprinting device to perform the following operations:

performing, prior to said identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks, a filtering operation on the audio signal to remove high frequency signals above a first frequency threshold level.

18 . The system of claim 15 , wherein said first processor further controls the audio fingerprinting device to perform the following operations:

quantizing the determined power spectral density (PSD) values of the audio signal.

19 . The system of claim 18 ,

wherein said operation of identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks includes:

generating a spectrogram of power spectral density values based on: (i) said determined power spectral density values of the audio signal, (ii) the set of frequency bins, and (iii) the plurality of time segments; and

applying a maximal filter to said spectrogram of power spectral density values to locate frequency peaks in said spectrogram.

20 . The system of claim 19 ,

wherein said operation of identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks further includes:

applying an erosion filter to said spectrogram of power spectral density values after applying said maximal filter.

21 . The system of claim 15 ,

wherein said audio signal is digitally encoded audio; and

wherein said first processor controls the audio fingerprinting device prior to determining, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins to perform the following operations:

decoding said digitally encoded audio; and

converting a sampling rate for said audio to an 8 KHz sampling rate when said sampling rate is not 8 KHz.

22 . A non-transitory computer readable medium including a first set of computer executable instructions which when executed by a processor of a computing device cause the computing device to:

remove silence from an audio signal, said audio signal being from a first call;

determine, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins, N being an integer greater than 1;

identify (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks;

generate a first audio fingerprint from at least some of the identified plurality of dominant frequency peaks and the identified positions in the audio signal corresponding to the identified peaks, said first audio fingerprint being an ordered set of information including: a first frequency bin value, a second frequency bin value, and a delta time, said first frequency bin value corresponding to a first dominant frequency peak, said first dominant frequency peak being one of the identified dominant frequency peaks, said second frequency bin value corresponding to a second dominant frequency peak, said second dominant frequency peak being one of the identified dominant frequency peaks, said first dominant frequency peak and said second dominant frequency peak being different dominant frequency peaks, said delta time being a time difference between a second identified location in the audio signal corresponding to the second dominant frequency peak and the first identified location in the audio signal corresponding to the first dominant frequency peak;

generate a first set of fuzzy audio fingerprints from the first audio fingerprint by modifying one or more of: the first frequency bin value or the second frequency bin value, said first frequency bin value identifying a first range of frequencies corresponding to a first frequency bin, said second frequency bin value identifying a second range of frequencies corresponding to a second frequency bin; and

use the first audio fingerprint and the first set of fuzzy audio fingerprints to determine whether the first call is a robocall; and

wherein said modifying includes performing one of the following:

(i) changing the first frequency bin value to a third frequency bin value, said third frequency bin value identifying a third range of frequencies corresponding to a third frequency bin, said third range of frequencies being different than said first range of frequencies,

(ii) changing the second frequency bin value to a fourth frequency bin value, said fourth frequency bin value identifying a fourth range of frequencies corresponding to a fourth frequency bin, said fourth range of frequencies being different than said second range of frequencies, or

(iii) changing the first frequency bin value to the third frequency bin value, said third frequency bin value identifying the third range of frequencies corresponding to the third frequency bin, said third range of frequencies being different than said first range of frequencies and changing the second frequency bin value to the fourth frequency bin value, said fourth frequency bin value identifying the fourth range of frequencies corresponding to the fourth frequency bin, said fourth range of frequencies being different than said second range of frequencies; and

wherein said plurality of N different frequency bins includes said first frequency bin, said second frequency bin, said third frequency bin, and said fourth frequency bin.

23 . A method of processing an audio signal comprising:

removing silence from the audio signal, said audio signal being from a first call;

after the silence has been removed from the audio signal, determining, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins, N being an integer greater than 1;

identifying (i) a plurality of dominant frequency peaks based on the determined power spectral density values, and (ii) positions in the audio signal corresponding to the identified peaks;

generating a first audio fingerprint from at least some of the identified plurality of dominant frequency peaks and the identified positions in the audio signal corresponding to the identified peaks, said first audio fingerprint being an ordered set of information including: a first frequency bin value, a second frequency bin value, and a delta time, said first frequency bin value corresponding to a first dominant frequency peak, said first dominant frequency peak being one of the identified dominant frequency peaks, said second frequency bin value corresponding to a second dominant frequency peak, said second dominant frequency peak being one of the identified dominant frequency peaks, said first dominant frequency peak and said second dominant frequency peak being different dominant frequency peaks, said delta time being a time difference between a second identified location in the audio signal corresponding to the second dominant frequency peak and the first identified location in the audio signal corresponding to the first dominant frequency peak;

generate a first set of fuzzy audio fingerprints from the first audio fingerprint, said generating the first set of fuzzy audio fingerprints from the first audio fingerprint including modifying one or more of the following of the first audio fingerprint: the first frequency bin value or the second frequency bin value; and

using the first audio fingerprint and the first set of fuzzy audio fingerprints to determine whether the first call is a robocall; and

wherein said generating the first set of fuzzy audio fingerprints from the first audio fingerprint including modifying one or more of the following of the first audio fingerprint: the first frequency bin value or the second frequency bin value includes:

generating a first fuzzy audio fingerprint and a second fuzzy audio fingerprint, said generating the first fuzzy audio fingerprint and the second fuzzy audio fingerprint including:

generating the first fuzzy audio fingerprint by adding 1 to the first frequency bin value when the first frequency bin value is a value greater than 1 and less than 64; and

generating the second fuzzy audio fingerprint by subtracting 1 from the first frequency bin value when the first frequency bin value is a value greater than 1 and less than 64; and

generating the first fuzzy audio fingerprint by adding 2 to the first frequency bin value when the first frequency bin value is a value equal to or greater than 64 and less than 128; and

generating the second fuzzy audio fingerprint by subtracting 2 from the first frequency bin value when the first frequency bin value is a value equal to or greater than 64 and less than 128; and

generating the first fuzzy audio fingerprint by adding 4 to the first frequency bin value when the first frequency bin value is a value equal to or greater than 128 and less than 256; and

generating the second fuzzy audio fingerprint by subtracting 4 from the first frequency bin value when the first frequency bin value is a value equal to or greater than 128 and less than 256.

24 . The method of claim 23 ,

wherein a 1024 point Fast Fourier Transform (FFT) is utilized for said determining, for a plurality of time segments of the audio signal, power spectral density values of the audio signal for each of a plurality of N different frequency bins;

wherein each of the time segments of the audio signal have been sampled at an 8000 Hz sample rate;

wherein the first frequency bin value is a quantized frequency value in the range of (0, 256) corresponding to 0-2000 Hz; and

wherein the second frequency bin value is a quantized frequency value in the range of (0, 256) corresponding to 0-2000 Hz.

Assignments (3)

Continuity (2)

References Cited (50)