System and method for automatic verification of the understandability of speech
View Patent ↗Disclosed herein are systems, methods, and computer-readable storage media for processing a message received from a user to determine whether an estimate of intelligibility is below an intelligibility threshold. The method includes recognizing a portion of a user's message that contains the one or more expected utterances from a critical information list, calculating an estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected utterances, and prompting the user to repeat at least the recognized portion of the user's message if the calculated estimate of intelligibility for the recognized portion of the user's message is below an intelligibility threshold. In one aspect, the method further includes prompting the user to repeat at least a portion of the message if any of a measured speech level and a measured signal-to-noise ratio of the user's message are determined to be below their respective thresholds.
1. A method comprising:
receiving speech in a first language;
recognizing, via a processor, at least one phoneme within the speech with reference to a list of phonemes in the first language, to yield a recognized portion;
determining, based on the recognized portion and a phoneme distribution for the first language, an estimate of intelligibility;
if the estimate of intelligibility is below a threshold, applying a phoneme distribution for a second language to determine an estimated language; and
communicating to a user at least one of the estimate of intelligibility and the estimated language.
2. The method of claim 1 , further comprising if the estimate of intelligibility is above the threshold determining other words within the speech using the phoneme distribution for the first language.
3. The method of claim 1 , further comprising if the estimate of intelligibility is below the threshold requesting the user repeat the speech.
4. The method of claim 1 , wherein the phoneme distribution for the first language is a domain-specific phoneme distribution.
5. The method of claim 4 , further comprising if the estimate of intelligibility is above the threshold assigning a domain to the speech, wherein the domain is associated with the domain-specific phoneme distribution.
6. The method of claim 4 , wherein the estimate of intelligibility is based on a number of words within the speech found in the domain-specific phoneme distribution.
7. The method of claim 1 , wherein the first language is English and the second language is one of Spanish, German, French, Japanese, Chinese, and Portuguese.
8. A system comprising:
a processor;
a storage device storing instructions for controlling the processor to perform steps comprising:
receiving speech in a first language;
recognizing at least one phoneme within the speech with reference to a list of phonemes in the first language, to yield a recognized portion;
determining, based on the recognized portion and a phoneme distribution for the first language, an estimate of intelligibility;
if the estimate of intelligibility is below a threshold, applying a phoneme distribution for a second language to determine an estimated language; and
communicating to a user at least one of the estimate of intelligibility and the estimated language.
9. The system of claim 8 , the steps further comprising if the estimate of intelligibility is above the threshold determining other words within the speech using the phoneme distribution for the first language.
10. The system of claim 8 , the steps further comprising if the estimate of intelligibility is below the threshold requesting the user repeat the speech.
11. The system of claim 8 , wherein the phoneme distribution for the first language is a domain-specific phoneme distribution.
12. The system of claim 11 , the steps, further comprising if the estimate of intelligibility is above the threshold assigning a domain to the speech, wherein the domain is associated with the domain-specific phoneme distribution.
13. The system of claim 11 , wherein the estimate of intelligibility is based on a number of words within the speech found in the domain-specific phoneme distribution.
14. The system of claim 8 , wherein the first language is English and the second language is one of Spanish, German, French, Japanese, Chinese, and Portuguese.
15. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform steps comprising:
receiving speech in a first language;
recognizing at least one phoneme within the speech with reference to a list of phonemes in the first language, to yield a recognized portion;
determining, based on the recognized portion and a phoneme distribution for the first language, an estimate of intelligibility;
if the estimate of intelligibility is below a threshold, applying a phoneme distribution for a second language to determine an estimated language; and
communicating to a user at least one of the estimate of intelligibility and the estimated language.
16. The non-transitory computer-readable storage medium of claim 15 , the instructions further comprising if the estimate of intelligibility is above the threshold determining other words within the speech using the phoneme distribution for the first language.
17. The non-transitory computer-readable storage medium of claim 15 , the instructions further comprising if the estimate of intelligibility is below the threshold requesting the user repeat the speech.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the phoneme distribution for the first language is a domain-specific phoneme distribution.
19. The non-transitory computer-readable storage medium of claim 18 , the instructions further comprising if the estimate of intelligibility is above the threshold assigning a domain to the speech, wherein the domain is associated with the domain-specific phoneme distribution.
20. The non-transitory computer-readable storage medium of claim 18 , wherein the estimate of intelligibility is based on a number of words within the speech found in the domain-specific phoneme distribution.