System and method for automatic detection of abnormal stress patterns in unit selection synthesis
View Patent ↗Disclosed herein are systems, methods, and non-transitory computer-readable storage media for detecting and correcting abnormal stress patterns in unit-selection speech synthesis. A system practicing the method detects incorrect stress patterns in selected acoustic units representing speech to be synthesized, and corrects the incorrect stress patterns in the selected acoustic units to yield corrected stress patterns. The system can further synthesize speech based on the corrected stress patterns. In one aspect, the system also classifies the incorrect stress patterns using a machine learning algorithm such as a classification and regression tree, adaptive boosting, support vector machine, and maximum entropy. In this way a text-to-speech unit selection speech synthesizer can produce more natural sounding speech with suitable stress patterns regardless of the stress of units in a unit selection database.
1. A method comprising:
receiving a stress pattern for both a language and an accent in the language, the stress pattern being associated with words in the language;
detecting, based on the stress pattern, incorrect stress patterns in selected acoustic units representing speech to be synthesized, wherein the selected acoustic units were selected by a separate unit-selection speech synthesizer; and
modifying, via a processor and prior to waveform synthesis, the incorrect stress patterns in the selected acoustic units to yield corrected stress patterns, wherein the corrected stress patterns conform to the stress pattern for the language, and wherein the detecting, based on the stress pattern, of incorrect stress patterns in the selected acoustic units and the modifying the incorrect stress patterns further comprise:
performing a word level detection of incorrect word stress patterns in the selected acoustic units;
modifying the incorrect word stresses to conform to the stress pattern, to yield word corrected stress patterns;
performing a phrase level detection of incorrect phrase stresses on the word corrected stress patterns;
modifying the incorrect phrase stresses to conform to the stress pattern, to yield phrase corrected stress patterns;
performing a sentence level detection of incorrect sentence stresses on the phrase corrected stress patterns; and
modifying the incorrect sentence stresses, to yield the corrected stress patterns.
2. The method of claim 1 , further comprising synthesizing speech based on the corrected stress patterns.
3. The method of claim 1 , wherein modifying the incorrect stress patterns occurs before waveform synthesis of the selected acoustic units.
4. The method of claim 1 , wherein the stress pattern comprises one of lexical stress, sentential stress, primary stress, and secondary stress.
5. The method of claim 1 , wherein detecting the incorrect stress patterns is based on detecting a difference in one of pitch, amplitude, and duration.
6. The method of claim 5 , wherein detecting the incorrect stress patterns is further based on a comparison of a selected acoustic unit with neighboring selected acoustic units.
7. The method of claim 1 , wherein modifying the incorrect stress patterns occurs in post-processing of a text-to-speech system.
8. The method of claim 1 , further comprising classifying the incorrect stress patterns.
9. The method of claim 8 , wherein a machine learning algorithm classifies the incorrect stress patterns.
10. The method of claim 9 , wherein the machine learning algorithm is one of a classification and regression tree, adaptive boosting, a support vector machine, and maximum entropy.
11. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
receiving a stress pattern for both a language and an accent in the language, the stress pattern being associated with words in the language;
detecting, based on the stress pattern, incorrect stress patterns in selected acoustic units representing speech to be synthesized, wherein the selected acoustic units were selected by a separate unit-selection speech synthesizer; and
modifying, via a processor and prior to waveform synthesis, the incorrect stress patterns in the selected acoustic units to yield corrected stress patterns, wherein the corrected stress patterns conform to the stress pattern for the language, and wherein the detecting, based on the stress pattern, of incorrect stress patterns in the selected acoustic units and the modifying the incorrect stress patterns further comprise:
performing a word level detection of incorrect word stress patterns in the selected acoustic units;
modifying the incorrect word stresses to conform to the stress pattern, to yield word corrected stress patterns;
performing a phrase level detection of incorrect phrase stresses on the word corrected stress patterns;
modifying the incorrect phrase stresses to conform to the stress pattern, to yield phrase corrected stress patterns;
performing a sentence level detection of incorrect sentence stresses on the phrase corrected stress patterns; and
modifying the incorrect sentence stresses, to yield the corrected stress patterns.
12. The system of claim 11 , the computer-readable storage medium having additional instructions stored which result in the operations further comprising synthesizing speech based on the corrected stress patterns.
13. The system of claim 11 , wherein modifying the incorrect stress patterns occurs before waveform synthesis of the selected acoustic units.
14. The system of claim 11 , wherein the stress pattern comprises one of lexical stress, sentential stress, primary stress, and secondary stress.
15. The system of claim 11 , wherein detecting the incorrect stress patterns is based on one of pitch, amplitude, and duration.
16. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving a stress pattern for both a language and an accent in the language, the stress pattern being associated with words in the language;
detecting, based on the stress pattern, incorrect stress patterns in selected acoustic units representing speech to be synthesized, wherein the selected acoustic units were selected by a separate unit-selection speech synthesizer; and
modifying, via a processor and prior to waveform synthesis, the incorrect stress patterns in the selected acoustic units to yield corrected stress patterns, wherein the corrected stress patterns conform to the stress pattern for the language, and wherein the detecting, based on the stress pattern, of incorrect stress patterns in the selected acoustic units and the modifying the incorrect stress patterns further comprise:
performing a word level detection of incorrect word stress patterns in the selected acoustic units;
modifying the incorrect word stresses to conform to the stress pattern, to yield word corrected stress patterns;
performing a phrase level detection of incorrect phrase stresses on the word corrected stress patterns;
modifying the incorrect phrase stresses to conform to the stress pattern, to yield phrase corrected stress patterns;
performing a sentence level detection of incorrect sentence stresses on the phrase corrected stress patterns; and
modifying the incorrect sentence stresses, to yield the corrected stress patterns.
17. The computer-readable storage device of claim 16 , having additional instructions stored which result in the operations further comprising synthesizing speech based on the corrected stress patterns.
18. The computer-readable storage device of claim 16 , wherein modifying the incorrect stress patterns occurs in post-processing of a text-to-speech system.
19. The method of claim 16 , wherein the performing of the word level detection, the performing of the phrase level detection, and the performing of the sentence level detection occur in parallel and over multiple passes.