Robust audio identification with interference cancellation
Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.
1. A method of audio speech recognition, comprising:
receiving an audio signal;
generating a spectrogram of the audio signal and a plurality of transform coefficients;
processing, using at least one of a hidden Markov model (HMM) or a pattern matching model, the plurality of transform coefficients to detect at least one of a first plurality of speech phonemes or human speech;
determining, based on the at least one of the first plurality of speech phonemes or human speech, one or more components of the spectrogram and one or more of the plurality of transform coefficients associated with likely interfering speech from an interference source;
subtracting, from the spectrogram, the one or more components associated with the likely interfering speech to form a clean audio signal; and
processing, using the at least one of the HMM or the pattern matching model, the clean audio signal to detect a second plurality of speech phonemes.
2. The method of claim 1 , wherein processing the clean audio signal comprises computing the HMM on the clean audio signal to detect the second plurality of speech phonemes.
3. The method of claim 2 , further comprising performing, using the HMM, speech recognition on the second plurality of speech phonemes.
4. The method of claim 1 , wherein processing the clean audio signal comprises computing a state based model on the clean audio signal to detect the second plurality of speech phonemes.
5. The method of claim 1 , wherein processing the clean audio signal comprises computing the pattern matching model on the clean audio signal to detect the second plurality of speech phonemes.
6. The method of claim 1 , wherein the spectrogram and the plurality of transform coefficients include a fundamental frequency and a plurality of formants.
7. The method of claim 1 , further comprising synthesizing the likely interfering speech from the spectrogram and the plurality of transform coefficients of the interference source.
8. The method of claim 1 , further comprising:
identifying, from the audio signal and using a plurality of microphones, a plurality of audio sources; and
separating the plurality of audio sources from audio signal.
9. The method of claim 1 , further comprising segmenting the audio signal into a plurality of separate sources for (i) speech and (ii) music.
10. A non-transitory machine-readable medium having instructions embodied thereon, which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
receiving an audio signal;
generating a spectrogram of the audio signal;
processing, using at least one of a hidden Markov model (HMM) or a pattern matching model, the plurality of transform coefficients to detect at least one of a first plurality of speech phonemes or human speech;
determining, based on the at least one of the first plurality of speech phonemes or human speech, one or more components of the spectrogram and one or more of the plurality of transform coefficients associated with likely interfering speech from an interference source;
subtracting, from the spectrogram, the one or more components associated with the likely interfering speech to form a clean audio signal; and
processing, using the at least one of the HMM or the pattern matching model, the clean audio signal to detect a second plurality of speech phonemes.
11. The non-transitory machine-readable medium of claim 10 , wherein processing the clean audio signal comprises computing the HMM on the clean audio signal to detect the second plurality of speech phonemes.
12. The non-transitory machine-readable medium of claim 11 , further comprising performing, using the HMM, speech recognition on the second plurality of speech phonemes.
13. The non-transitory machine-readable medium of claim 10 , wherein processing the clean audio signal comprises computing a state based model on the clean audio signal to detect the second plurality of speech phonemes.
14. The non-transitory machine-readable medium of claim 10 , wherein processing the clean audio signal comprises computing the pattern matching model on the clean audio signal to detect the second plurality of speech phonemes.
15. The non-transitory machine-readable medium of claim 10 , wherein the spectrogram and the plurality of transform coefficients include a fundamental frequency and a plurality of formants.
16. The non-transitory machine-readable medium of claim 10 , further comprising synthesizing the likely interfering speech from the spectrogram and the plurality of transform coefficients of the interference source.
17. The non-transitory machine-readable medium of claim 10 , further comprising:
identifying, from the audio signal and using a plurality of microphones, a plurality of audio sources; and
separating the plurality of audio sources from audio signal.
18. A speech recognition system, comprising:
a receiver configured to receive an audio signal;
a memory that stores instructions; and
one or more processors configured by the instructions to perform operations comprising:
generating a spectrogram of the audio signal;
processing, using at least one of a hidden Markov model (HMM) or a pattern matching model, the plurality of transform coefficients to detect at least one of a first plurality of speech phonemes or human speech;
determining, based on the at least one of the first plurality of speech phonemes or human speech, one or more components of the spectrogram and one or more of the plurality of transform coefficients associated with likely interfering speech from an interference source;
subtracting, from the spectrogram, the one or more components associated with the likely interfering speech to form a clean audio signal; and
processing, using the at least one of the HMM or the pattern matching model, the clean audio signal to detect a second plurality of speech phonemes.
19. The speech recognition system of claim 18 , wherein processing the clean audio signal comprises computing the HMM on the clean audio signal to detect the second plurality of speech phonemes.
20. The speech recognition system of claim 18 , wherein processing the clean audio signal comprises computing the pattern matching model on the clean audio signal to detect the second plurality of speech phonemes.