Method and apparatus for classifying lexical stress

1. A method for classifying lexical stress in a speech sample to enable a computing device to provide stress pronunciation feedback usable by a speaker whose speech is represented in the speech sample, the method comprising:

determining a plurality of syllables in the speech sample;

locating vowels associated with the syllables of the speech sample;

executing a feature extractor on the computing system;

with the feature extractor, over a duration of a vowel located in a syllable of the speech sample, determining a plurality of features of the vowel that are usable by the computing system to determine a level of stress associated with the syllable;

executing a modeling engine on the computing system;

with the modeling engine, using the features of the vowel and at least one model created using other speech samples, computing a plurality of scores for the vowel relating to at least two of a likelihood that the syllable is primary stressed and a likelihood that the syllable is secondary stressed and a likelihood that the syllable is unstressed;

executing a classifier on the computing system;

using as inputs to the classifier a threshold, the plurality of scores for the vowel, and a canonical stress level for the syllable, determining a stress label for the syllable;

tuning the threshold to improve stress pronunciation feedback output by the computing device in response to the speech sample.

2. The method of claim 1 , wherein determining the plurality of features of the vowel comprises computing a segmental feature on the vowel.

3. The method of claim 2 , wherein determining the plurality of features of the vowel comprises computing a plurality of spectral features over a time frame associated with the vowel.

4. The method of claim 3 , comprising including both the segmental features and the spectral features in a feature vector.

5. The method of claim 4 , comprising using the feature vector to compute the plurality of scores.

6. The method of claim 1 , wherein the plurality of scores comprises at least three scores relating to a likelihood that the syllable is primary stressed and a likelihood that the syllable is secondary stressed and a likelihood that the syllable is unstressed.

7. The method of claim 1 , wherein the threshold comprises a plurality of thresholds corresponding to the at least two of the likelihood that the syllable is primary stressed and the likelihood that the syllable is secondary stressed and the likelihood that the syllable is unstressed.

8. The method of claim 1 , comprising outputting the stress pronunciation feedback to an output device.

9. The method of claim 1 , comprising outputting the stress pronunciation feedback for use by a language learning system.

10. A system for enabling a computing device to interpret an un-interpreted portion of natural language captured by an audio input device coupled to the computing device so that the computing device can execute an action in response to the un-interpreted portion of the natural language, the system comprising:

one or more processors;

a communication interface coupled to the one or more processors;

one or more non-transitory computer-readable storage media coupled to the one or more processors and storing sequences of instructions, which when executed by the one or more processors, cause the one or more processors to perform operations comprising:

determining a plurality of syllables in the speech sample;

locating vowels associated with the syllables of the speech sample;

executing a feature extractor on the computing system;

with the feature extractor, over a duration of a vowel located in a syllable of the speech sample, determining a plurality of features of the vowel that are usable by the computing system to determine a level of stress associated with the syllable;

executing a modeling engine on the computing system;

with the modeling engine, using the features of the vowel and at least one model created using other speech samples, computing a plurality of scores for the vowel relating to at least two of a likelihood that the syllable is primary stressed and a likelihood that the syllable is secondary stressed and a likelihood that the syllable is unstressed;

executing a classifier on the computing system;

using as inputs to the classifier a threshold, the plurality of scores for the vowel, and a canonical stress level for the syllable, determining a stress label for the syllable;

tuning the threshold to improve stress pronunciation feedback output by the computing device in response to the speech sample.

11. The system of claim 10 , wherein determining the plurality of features of the vowel comprises computing a segmental feature on the vowel.

12. The system of claim 11 , wherein determining the plurality of features of the vowel comprises computing a plurality of spectral features over a time frame associated with the vowel.

13. The system of claim 12 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising including both the segmental features and the spectral features in a feature vector.

14. The system of claim 13 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising using the feature vector to compute the plurality of scores.

15. The system of claim 10 , wherein the plurality of scores comprises at least three scores relating to a likelihood that the syllable is primary stressed and a likelihood that the syllable is secondary stressed and a likelihood that the syllable is unstressed.

16. The system of claim 10 , wherein the threshold comprises a plurality of thresholds corresponding to the at least two of the likelihood that the syllable is primary stressed and the likelihood that the syllable is secondary stressed and the likelihood that the syllable is unstressed.

17. The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising outputting the stress pronunciation feedback to an output device.

18. The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising outputting the stress pronunciation feedback for use by a language learning system.

19. A computer program product for enabling a computing device to interpret an un-interpreted portion of natural language captured by an audio input device coupled to the computing device so that the computing device can execute an action in response to the un-interpreted portion of the natural language, the computer program product comprising one or more non-transitory computer readable storage media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

determining a plurality of syllables in the speech sample;

locating vowels associated with the syllables of the speech sample;

executing a feature extractor on the computing system;

with the feature extractor, over a duration of a vowel located in a syllable of the speech sample, determining a plurality of features of the vowel that are usable to determine a level of stress associated with the syllable;

executing a modeling engine on the computing system;

with the modeling engine, using the features of the vowel and at least one model created using other speech samples, computing a plurality of scores for the vowel relating to at least two of a likelihood that the syllable is primary stressed and a likelihood that the syllable is secondary stressed and a likelihood that the syllable is unstressed;

executing a classifier on the computing system;

using as inputs to the classifier a threshold, the plurality of scores for the vowel, and a canonical stress level for the syllable, determining a stress label for the syllable;

tuning the threshold to improve stress pronunciation feedback output by the computing device in response to the speech sample.

20. The computer program product of claim 19 , wherein determining the plurality of features of the vowel comprises computing a segmental feature on the vowel.

21. The computer program product of claim 20 , wherein determining the plurality of features of the vowel comprises computing a plurality of spectral features over a time frame associated with the vowel.

22. The computer program product of claim 21 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising including both the segmental features and the spectral features in a feature vector.

23. The computer program product of claim 22 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising using the feature vector to compute the plurality of scores.

24. The computer program product of claim 19 , wherein the plurality of scores comprises at least three scores relating to a likelihood that the syllable is primary stressed and a likelihood that the syllable is secondary stressed and a likelihood that the syllable is unstressed.

25. The computer program product of claim 19 , wherein the threshold comprises a plurality of thresholds corresponding to the at least two of the likelihood that the syllable is primary stressed and the likelihood that the syllable is secondary stressed and the likelihood that the syllable is unstressed.

26. The computer program product of claim 19 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising outputting the stress pronunciation feedback to an output device or for use by a language learning system.