Robust audio identification with interference cancellation
Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.
1. A method of content identification, the method comprising:
identifying, by a processor and in a query, a first segment of query signatures in a sliding window that has a high match score with reference signatures of a reference and a second segment of query signatures in the sliding window that has a low match score with the reference signatures, wherein the second segment represents a distorted segment, wherein the query is based on an audio signal captured by a microphone;
evaluating, by the processor, query matches using the first segment with the high match score and not using the second segment with the low match score;
outputting, by the processor, a matching reference and match scores; and
performing a second analysis on a remote server storing more information on the reference than is stored on a portable smart device performing the method and having an acoustic profile of the query sent by the portable smart device to the server to identify different and finer segments for interfering sources.
2. The method of claim 1 further comprising:
performing a statistical analysis of the second segment with the low match score to detect distortion patterns; and
determining, based on the statistical analysis and the detected distortion patterns, that compensation of the second segment with the low match score is not possible.
3. The method of claim 1 further comprising:
performing a statistical analysis of the first segment with the high match score to detect distortion patterns;
determining, based on the statistical analysis and the detected distortion patterns, that compensation of the first segment with the high match score is possible; and
compensating the first segment with the high match score for the detected distortion patterns.
4. The method of claim 1 further comprising:
repeating the identifying and the evaluating for a new position of the sliding window after a pre-specified gap between a first position and the new position to determine a better match.
5. The method of claim 1 , wherein detected features are used to increase a length of a hash key for reference signature indexing to detect more likely match candidates for a query.
6. The method of claim 1 , further comprising outputting,
based on the evaluation of the query matches using the first segment with the high match score, text for speech in the audio signal.
7. The method of claim 1 , further comprising:
capturing, using the microphone, the audio signal; and
generating, based on the audio signal, the query signatures.
8. A non-transitory computer readable storage medium comprising instructions that, when executed, cause a machine to perform operations comprising:
identify, in a query, a first segment of query signatures in a sliding window that has a high match score with reference signatures of a reference and a second segment of query signatures in the sliding window that has a low match score with the reference signatures, wherein the second segment represents a distorted segment, wherein the query is based on an audio signal captured by a microphone;
evaluate query matches using the first segment with the high match score and not using the second segment with the low match score;
output a matching reference and match scores; and
perform a second analysis on a remote server storing more information on the reference than is stored on a portable smart device including the non-transitory computer readable medium and having an acoustic profile of the query sent by the portable smart device to the server to identify different and finer segments for interfering sources.
9. The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the machine to:
perform a statistical analysis of the second segment with the low match score to detect distortion patterns; and
determine, based on the statistical analysis and the detected distortion patterns, that compensation of the second segment with the low match score is not possible.
10. The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the machine to perform a statistical analysis of the first segment with the high match score to detect distortion patterns;
determine, based on the statistical analysis and the detected distortion patterns, that compensation of the first segment with the high match score is possible; and
compensate the first segment with the high match score for the detected distortion patterns.
11. The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the machine to repeat the identifying and the evaluating for a new position of the sliding window after a pre-specified gap between a first position and the new position to determine a better match.
12. The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the machine to use detected features to increase a length of a hash key for reference signature indexing to detect more likely match candidates for a query.
13. An apparatus comprising:
at least one processor; and
a memory in communication with the at least one processor, the memory including a non-transitory computer-readable medium comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
identify, in a query, a first segment of query signatures in a sliding window that has a high match score with reference signatures of a reference and a second segment of query signatures in the sliding window that has a low match score with the reference signatures, wherein the second segment represents a distorted segment, wherein the query is based on an audio signal captured by a microphone;
evaluate query matches using the first segment with the high match score and not using the second segment with the low match score;
output a matching reference and match scores; and
perform a second analysis on a remote server storing more information on the reference than is stored on the apparatus and having an acoustic profile of the query sent by the apparatus to the server to identify different and finer segments for interfering sources.
14. The apparatus of claim 13 , wherein the instructions, when executed, cause the at least one processor to:
perform a statistical analysis of the second segment with the low match score to detect distortion patterns; and
determine, based on the statistical analysis and the detected distortion patterns, that compensation of the second segment with the low match score is not possible.
15. The apparatus of claim 13 , wherein the instructions, when executed, cause the at least one processor to:
perform a statistical analysis of the first segment with the high match score to detect distortion patterns;
determine, based on the statistical analysis and the detected distortion patterns, that compensation of the first segment with the high match score is possible; and
compensate the first segment with the high match score for the detected distortion patterns.
16. The apparatus of claim 13 , wherein the instructions, when executed, cause the at least one processor to repeat the identifying and the evaluating for a new position of the sliding window after a pre-specified gap between a first position and the new position to determine a better match.
17. The apparatus of claim 13 , wherein the instructions, when executed, cause the at least one processor to use detected features to increase a length of a hash key for reference signature indexing to detect more likely match candidates for a query.