System and method for automatic verification of the understandability of speech
View Patent ↗The present invention relates to a system and method for automatically verifying that a message received from a user is intelligible. In an exemplary embodiment, a message is received from the user. A speech level of the user's message may be measured and compared to a pre-determined speech level threshold to determine whether the measured speech level is below the pre-determined speech level threshold. A signal-to-noise ratio of the user's message may be measured and compared to a pre-determined signal-to-noise ratio threshold to determine whether the measured signal-to-noise ratio of the message is below the pre-determined signal-to-noise ratio threshold. An estimate of intelligibility for the user's message may be calculated and compared to an intelligibility threshold to determine whether the calculated estimate of intelligibility is below the intelligibility threshold. If any of the measured speech level, measured signal-to-noise ratio and calculated estimate of intelligibility of the user's message are determined to be below their respective thresholds, the user may be prompted to repeat at least a portion of the message.
1. A computer-implemented method for processing a message received from a user to determine whether an estimate of intelligibility is below an intelligibility threshold, the method comprising:
determining via a processor whether any portion of the user's message contains one or more expected phonemes from a complete set of phonemes expected for a particular language;
accessing an extended set of phonemes that are not found in the particular language if one or more phonemes that are expected for the particular language are not recognized;
recognizing the portion of the user's message that contains the one or more expected phonemes;
recognizing in the user's message one or more phonemes from the extended set of phonemes;
calculating via the processor an estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes based on the recognized one or more phonemes;
comparing the estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes with the intelligibility threshold; and
if the calculated estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes is below the intelligibility threshold, prompting the user to repeat at least the recognized portion of the user's message.
2. The method of claim 1 , further comprising:
prompting the user to repeat at least a portion of the message if any of a measured speech level and a measured signal-to-noise ratio of the user's message are determined to be below their respective thresholds.
3. The method of claim 1 , further comprising:
combining the recognized phonemes into a single text string; and
parsing the single text string into recognizable words.
4. The method of claim 1 , further comprising:
analyzing a phoneme distribution based on the recognized one or more phonemes from the extended set of phonemes; and
based on the analyzed phoneme distribution, identifying which language is being spoken by the user.
5. A method determining whether a calculated estimate of intelligibility for a received user message is below an intelligibility, the method comprising:
determining a domain for the user's message;
accessing a domain library from a plurality of domain libraries corresponding to the determined domain;
calculating via a processor a percentage of words in the user's message that correlate to words in the accessed domain library;
comparing the calculated percentage of words in the user's message that correlate to words in the accessed domain library with the intelligibility threshold; and
if the calculated percentage of words in the user's message that correlate to words in the accessed domain library is below the intelligibility threshold, prompting the user to repeat at least a portion of the message.
6. The method of claim 5 , further comprising:
prompting the user to repeat at least a portion of the message if any of a measured speech level and a measured signal-to-noise ratio of the user's message are determined to be below their respective thresholds.
7. The method of claim 6 , wherein determining the domain of the user's message comprises:
recognizing the words included in the user's message;
determining which of the plurality of domain libraries contains one or more of the recognized words; and
selecting one or more domain libraries that contain the one or more of the recognized words.
8. The method of claim 6 , wherein determining the domain of the user's message comprises:
prompting the user for information relating to the domain of the user's message.
9. The method of claim 6 , further comprising:
if any of a measured speech level and a measured signal-to-noise ratio of the user's message are determined to be below their respective thresholds, replaying at least the portion of the message back to the user.
10. A system for processing a message received from a user to determine whether an estimate of intelligibility is below an intelligibility threshold, the system comprising:
a processor;
a determination module configured to control the processor to determine whether any portion of the user's message contains one or more expected phonemes from a complete set of phonemes expected for a particular language;
an access module configured to control the processor to access an extended set of phonemes that are not found in the particular language if one or more phonemes that are expected for the particular language are not recognized;
a recognition module configured to control the processor to recognize the portion of the user's message that contains the one or more expected phonemes;
a second recognition module configured to control the processor to recognize in the user's message one or more phonemes from the extended set of phonemes;
a calculation module configured to control the processor to calculate an estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes based on the recognized one or more phonemes;
a comparison module configured to control the processor to compare the estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes with the intelligibility threshold; and
a prompting module configured, if the calculated estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes is below the intelligibility threshold, to control the processor to prompt the user to repeat at least the recognized portion of the user's message.
11. The system of claim 10 , further comprising:
a repeating module configured to control the processor to prompt the user to repeat at least a portion of the message if any of a measured speech level and a measured signal-to-noise ratio of the user's message are determined to be below their respective thresholds.
12. A computer readable medium storing instructions for controlling a computing device to process a message received from a user to determine whether an estimate of intelligibility is below an intelligibility threshold, the instructions comprising:
determining whether any portion of the user's message contains one or more expected phonemes from a complete set of phonemes expected for a particular language;
accessing an extended set of phonemes that are not found in the particular language if one or more phonemes that are expected for the particular language are not recognized;
recognizing the portion of the user's message that contains the one or more expected phonemes;
recognizing in the user's message one or more phonemes from the extended set of phonemes;
calculating an estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes based on the recognized one or more phonemes;
comparing the estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes with the intelligibility threshold; and
if the calculated estimate of intelligibility for the recognized portion of the user's message that contains the one or more expected phonemes is below the intelligibility threshold, prompting the user to repeat at least the recognized portion of the user's message.
13. The computer readable medium of claim 12 , the instructions further comprising:
if any of the measured speech level, measured signal-to-noise ratio and calculated estimate of intelligibility of the user's message are determined to be below their respective thresholds, replaying at least the portion of the message back to the user.
14. The computer readable medium of claim 13 , the instructions further comprising:
prompting the user to repeat at least a portion of the message if any of a measured speech level and a measured signal-to-noise ratio of the user's message are determined to be below their respective thresholds.
15. A computer readable medium storing instructions for controlling a computing device to process a message received from a user to determine whether a calculated estimate of intelligibility for a received user message is below an intelligibility, the instructions comprising:
determining a domain for the user's message;
accessing a domain library from a plurality of domain libraries corresponding to the determined domain;
calculating a percentage of words in the user's message that correlate to words in the accessed domain library;
comparing the calculated percentage of words in the user's message that correlate to words in the accessed domain library with the intelligibility threshold; and
if the calculated percentage of words in the user's message that correlate to words in the accessed domain library is below the intelligibility threshold, prompting the user to repeat at least a portion of the message.
16. The computer-readable medium of claim 15 , the instructions further comprising:
recognizing the words included in the user's message;
determining which of the plurality of domain libraries contains one or more of the recognized words; and
selecting one or more domain libraries that contain the one or more of the recognized words.
17. The computer-readable medium of claim 15 , the instructions further comprising:
prompting the user for information relating to the domain of the user's message.