System and method for automatic detection of abnormal stress patterns in unit selection synthesis
View Patent ↗Disclosed herein are systems, methods, and non-transitory computer-readable storage media for detecting and correcting abnormal stress patterns in unit-selection speech synthesis. A system practicing the method detects incorrect stress patterns in selected acoustic units representing speech to be synthesized, and corrects the incorrect stress patterns in the selected acoustic units to yield corrected stress patterns. The system can further synthesize speech based on the corrected stress patterns. In one aspect, the system also classifies the incorrect stress patterns using a machine learning algorithm such as a classification and regression tree, adaptive boosting, support vector machine, and maximum entropy. In this way a text-to-speech unit selection speech synthesizer can produce more natural sounding speech with suitable stress patterns regardless of the stress of units in a unit selection database.
1. A method comprising:
receiving a stress pattern for both a language and an accent in the language;
detecting, according to the stress pattern, incorrect stress patterns in selected acoustic units representing speech to be synthesized, wherein the selected acoustic units were selected by a separate unit-selection speech synthesizer;
performing an analysis of the incorrect stress patterns, wherein the analysis comprises a word level analysis, a phrase level analysis, and a sentence level analysis on the incorrect stress patterns, wherein the word level analysis, the phrase level analysis, and the sentence level analysis are performed in series; and
modifying, via a processor and prior to waveform synthesis, the incorrect stress patterns in the selected acoustic units according to the analysis, to yield corrected stress patterns, wherein the corrected stress patterns conform to the stress pattern for the language.
2. The method of claim 1 , wherein the word level analysis, the phrase level analysis, and the sentence level analysis are performed in parallel.
3. The method of claim 1 , wherein the detecting of incorrect stress patterns, the performing of the analysis of the incorrect stress patterns, and the modifying of the incorrect stress patterns is performed on individual words.
4. The method of claim 1 , wherein the detecting of incorrect stress patterns, the performing of the analysis of the incorrect stress patterns, and the modifying of the incorrect stress patterns is performed on phrases.
5. The method of claim 1 , wherein the detecting of incorrect stress patterns, the performing of the analysis of the incorrect stress patterns, and the modifying of the incorrect stress patterns is performed on sentences.
6. The method of claim 1 , further comprising synthesizing speech according to the corrected stress patterns.
7. The method of claim 1 , wherein modifying the incorrect stress patterns occurs before waveform synthesis of the selected acoustic units.
8. The method of claim 1 , wherein the stress pattern comprises one of lexical stress, sentential stress, primary stress, and secondary stress.
9. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, result in the processor performing operations comprising:
receiving a stress pattern for both a language and an accent in the language;
detecting, according to the stress pattern, incorrect stress patterns in selected acoustic units representing speech to be synthesized, wherein the selected acoustic units were selected by a separate unit-selection speech synthesizer;
performing an analysis of the incorrect stress patterns, wherein the analysis comprises a word level analysis, a phrase level analysis, and a sentence level analysis on the incorrect stress patterns, wherein the word level analysis, the phrase level analysis, and the sentence level analysis are performed in series; and
modifying, via a processor and prior to waveform synthesis, the incorrect stress patterns in the selected acoustic units according to the analysis, to yield corrected stress patterns, wherein the corrected stress patterns conform to the stress pattern for the language.
10. The system of claim 9 , wherein the detecting of incorrect stress patterns, the performing of the analysis of the incorrect stress patterns, and the modifying of the incorrect stress patterns is performed on individual words.
11. The system of claim 9 , wherein the detecting of incorrect stress patterns, the performing of the analysis of the incorrect stress patterns, and the modifying of the incorrect stress patterns is performed on phrases.
12. The system of claim 9 , wherein the detecting of incorrect stress patterns, the performing of the analysis of the incorrect stress patterns, and the modifying of the incorrect stress patterns is performed on sentences.
13. The system of claim 9 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising synthesizing speech according to the corrected stress patterns.
14. The system of claim 9 , wherein modifying the incorrect stress patterns occurs before waveform synthesis of the selected acoustic units.
15. The system of claim 9 , wherein the stress pattern comprises one of lexical stress, sentential stress, primary stress, and secondary stress.
16. A computer-readable storage device having instructions stored which, when executed by a computing device, result in the computing device performing operations comprising:
receiving a stress pattern for both a language and an accent in the language;
detecting, according to the stress pattern, incorrect stress patterns in selected acoustic units representing speech to be synthesized, wherein the selected acoustic units were selected by a separate unit-selection speech synthesizer;
performing an analysis of the incorrect stress patterns, wherein the analysis comprises a word level analysis, a phrase level analysis, and a sentence level analysis on the incorrect stress patterns, wherein the word level analysis, the phrase level analysis, and the sentence level analysis are performed in series; and
modifying, via a processor and prior to waveform synthesis, the incorrect stress patterns in the selected acoustic units according to the analysis, to yield corrected stress patterns, wherein the corrected stress patterns conform to the stress pattern for the language.
17. The computer-readable storage device of claim 16 , having additional instructions stored which, when executed by the computing device, result in operations comprising synthesizing speech according to the corrected stress patterns.
18. The computer-readable storage device of claim 16 , wherein modifying the incorrect stress patterns occurs before waveform synthesis of the selected acoustic units.
19. The computer-readable storage device of claim 16 , wherein the stress pattern comprises one of lexical stress, sentential stress, primary stress, and secondary stress.