Audio search using codec frames
To detect events in an audio stream, frames of an audio signal (e.g., frames generated by a codec for a voice call or music stream) are received. Based on information in the frames, an index is used to look up an entry in a table associated with the codec. Each entry in the table indicates a likelihood that a frame matches a sound model element. The likelihood is used in the search for a sound bite, word, and/or phrase in the audio signal. The process of dynamic programming is used to find the combined likelihood for a match of the word, phrase, and/or sound bite to a region of the audio stream. Upon detection of the word, phrase, and/or sound bite in the audio stream, an event is generated, such as, notifying a person or logging the event in a database.
1. A method for analyzing an audio signal comprising:
receiving, by a microprocessor, a plurality of frames of the audio signal, wherein the audio signal is a live audio stream that was previously converted, by a type of codec, from an analog audio signal, into the plurality of frames of the audio signal;
determining, by the microprocessor, the type of the codec used to generate the plurality of frames;
in response to determining the type of codec used to generate the plurality of frames, selecting, by the microprocessor, a lookup table from a plurality of lookup tables based on the type of codec;
indexing, by the microprocessor, into a plurality of entries in the lookup table based on the plurality of frames, wherein the plurality of entries in the lookup table indicate a likelihood of the plurality of frames matching a plurality of sound model elements;
deriving, by the microprocessor, from the plurality of indexed lookup table entries, a match score for each of a plurality of sound bites, words, or phrases that are being searched for in the audio signal;
determining, by the microprocessor, that the match score is above a threshold; and
in response to determining that the match score is above the threshold, generating, by the microprocessor, an event.
2. The method of claim 1 , wherein the audio signal is a live voice communication between two communication devices and wherein the event is at least one of the following:
notifying an agent, via an agent communication device of a detection of the word or phrase;
notifying a supervisor, via a supervisor communication device of the detection of the word or phrase;
logging a detection of the event in a database;
conferencing a user communication device, of a new user into a voice/video call;
transferring a voice/video call to an Interactive Voice Response System (IVR);
forwarding/transferring a voice/video call to the user communication device;
searching for a new sound bite, word or phrase; or
switching to a different codec.
3. The method of claim 1 , wherein the plurality of entries in the lookup table comprise identifiers for sound model elements for a specific spoken language.
4. The method of claim 1 , wherein the audio signal is a voice communication between a user communication device and an Interactive Voice Response (IVR) system and wherein the event is at least one of:
retrieval of a document;
transferring of the voice communication; or
authentication of the user.
5. The method of claim 1 , further comprising at least one of:
removing, by the microprocessor, information from at least one of the plurality of frames that is not used in searching for the plurality of sound model elements;
ignoring, by the microprocessor, at least one of the plurality of frames that is deemed to be similar; or
ignoring, by the microprocessor, at least one of the plurality of frames that does not contain a level of sound.
6. The method of claim 1 , wherein the audio signal comprises a plurality of audio signals and further comprising:
detecting, by the microprocessor, one of the sound bites, words, or phrases at the same point in time in the plurality of audio signals; and
in response to detecting the one of the sound bites, words, or phrases at the same point in time in the plurality of audio signals, only storing, by the microprocessor, one of the plurality of audio signals.
7. A system for analyzing an audio signal comprising:
a microprocessor; and
a computer readable medium, coupled with the microprocessor and comprising microprocessor readable and executable instructions that program the microprocessor to execute:
an audio signal monitor that receives a plurality of frames of the audio signal, determines a type of the codec used to generate the plurality frames, wherein the audio signal was previously converted, by the type of codec, from an analog audio signal into the plurality of frames of the audio signal;
a sound analyzer that selects a lookup table from a plurality of lookup tables based on the type of codec in response to determining the type of codec used to generate the plurality of frames, indexes into a plurality of entries in the lookup table based on the plurality of frames, wherein the plurality of entries in the lookup table indicate a likelihood of the plurality of frames matching a plurality of sound model elements, derives, from the plurality of indexed lookup table entries, a match score for each of a plurality of sound bites, words, or phrases that are being searched for in the audio signal, and determines that the match score is above a threshold; and
an event detector that generates an event in response to determining the match score is above the threshold.
8. The system of claim 7 , wherein the audio signal is a live voice communication between two communication devices and wherein the event is at least one of the following:
notifying an agent, via an agent communication device of a detection of the word or phrase;
notifying a supervisor, via a supervisor communication device of the detection of the word or phrase;
logging a detection of the event in a database;
conferencing a new user communication device into a voice/video call;
transferring a voice/video call to an Interactive Voice Response System (IVR);
forwarding/transferring a voice/video call to the new user communication device;
searching for a new sound bite, word or phrase; or
switching to a different codec.
9. The system of claim 7 , wherein the plurality of entries in the lookup table comprise identifiers of the plurality of sound model elements for a specific spoken language.
10. The system of claim 7 , wherein the audio signal is a voice communication between a user communication device and an Interactive Voice Response system and wherein the event is at least one of:
retrieval of a document;
transferring of the voice communication; or
authentication of the user.
11. The system of claim 10 , wherein the event is the retrieval of the document.
12. The system of claim 7 , wherein the audio signal monitor does at least one of the following:
removes information from at least one of the plurality of frames that is not used in searching for the plurality of sound model elements;
ignores at least one of the plurality of frames that is deemed to be similar; or
ignores at least one of the plurality of frames that does not contain a level of sound.
13. The system of claim 7 , wherein the audio signal comprises a plurality of audio signals and wherein the sound analyzer:
detects one of the sound bites, words, or phrases at the same point in time in the plurality of audio signals; and
only stores one of the plurality of audio signals in response to detecting the one of the sound bites, words, or phrases at the same point in time in the plurality of audio signals.
14. A system for analyzing an audio signal comprising:
a microprocessor; and
a computer readable medium, coupled with the microprocessor and comprising microprocessor readable and executable instructions that program the microprocessor to execute:
an audio signal monitor that receives a plurality of frames of the audio signal, wherein the audio signal is converted, by a type of codec, from an analog signal into the plurality of frames, wherein the audio signal is a live voice communication between two communication devices on a network;
a sound analyzer that indexes into a plurality of entries in a lookup table based on the plurality of frames, wherein the plurality of entries in the lookup table indicate a likelihood of the plurality of frames matching a plurality of sound model elements, derives, from the plurality of indexed lookup table entries, a match score for each of a plurality of sound bites, words, or phrases that are being searched for in the audio signal, and determines that the match score is above a threshold; and
an event detector that generates an event in response to determining the match score is above the threshold, wherein the event is at least one of the following:
notifying an agent, via an agent communication device of a detection of the word or phrase;
notifying a supervisor, via a supervisor communication device of the detection of the word or phrase;
logging a detection of the event in a database;
conferencing a new user communication device into a voice/video call;
transferring a voice/video call to an Interactive Voice Response System (IVR);
forwarding/transferring a voice/video call to the new user communication device;
searching for a new sound bite, word or phrase; or
switching to a different codec.
15. The system of claim 14 , wherein the at least one event comprises notifying the agent communication device of the detection of the word or phrase.
16. The system of claim 14 , wherein the at least one event comprises switching to the different codec.
17. The system of claim 14 , wherein the at least one event comprises searching for the new sound bite, word, or phrase.
18. The system of claim 14 , wherein the at least one event comprises transferring the voice/video call to the IVR system.
19. The system of claim 14 , wherein the at least one event comprises forwarding/transferring the voice/video call to the new user communication device.
20. The system of claim 14 , wherein the at least one event comprises conferencing the new user communication device into the voice/video call.