Fast waveform synchronization for concentration and time-scale modification of speech
View Patent ↗A synthesis method for concatenative speech synthesis is provided for efficiently concatenating waveform segments in the time-domain. A digital waveform provider produces an input sequence of digital waveform segments. A waveform concatenator concatenates the input segments by using waveform blending within a concatenation zone to synchronize, weight, and overlap-add selected portions of the input segments to produce a single digital waveform. The synchronizing includes determining a minimum weighted energy anchor in the selected portion of each input segment and aligning synchronization peaks in a local vicinity of each anchor.
1. A digital waveform concatenation system for use in an acoustic processing application, the system comprising:
a digital waveform provider that produces an input sequence of at least two digital waveform segments, each waveform segment being a sequence of samples; and
a waveform concatenator that:
i. synchronizes input waveform segments to form a sequence of partially overlapping waveform segments, and
ii. weights and adds selected portions of the overlapping waveform segments to concatenate the input waveform segments so as to produce a single digital waveform;
wherein for segments of voiced speech, the synchronizing includes aligning a minimum energy anchor in each waveform segment with a corresponding minimum energy anchor of an adjacent waveform segment, each minimum energy anchor location in a given segment being optimized based on determining minimum weighted energy in a neighborhood of a boundary of the given segment.
2. A concatenation system according to claim 1 , wherein the acoustic processing application includes a text-to-speech application.
3. A concatenation system according to claim 1 , wherein the acoustic processing application includes a speech broadcast application.
4. A concatenation system according to claim 1 , wherein the acoustic processing application includes a carrier-slot application.
5. A concatenation system according to claim 1 , wherein the acoustic processing application includes a time-scale modification system.
6. A concatenation system according to claim 1 , wherein the waveform segments include at least one of speech diphones and speech triphones.
7. A concatenation system according to claim 1 , wherein the waveform segments include at least one of speech phones and speech demi-phones.
8. A concatenation system according to claim 1 , wherein the waveform segments include at least one of speech demi-syllables, speech syllables, words, and phrases.
9. A concatenation system according to claim 1 , wherein determining minimum weighted energy in the selected portion includes using a sliding weighted energy calculation algorithm.
10. A concatenation system according to claim 1 , wherein the input segments are filtered before synchronizing.
11. A concatenation system according to claim 1 , wherein aligning minimum energy anchors includes determining a largest waveform peak or trough in the close neighborhood of each minimum energy anchor.
12. A concatenation system according to claim 11 , wherein the close neighborhood is an interval of at least one pitch period containing the minimum energy anchor.
13. A concatenation system according to claim 11 , wherein the close neighborhood is the selected portion of the input segment.
14. A concatenation system according to claim 11 , wherein the location of one minimum energy anchor is the lowest weighted energy location in the selected portion.
15. A concatenation system according to claim 14 , wherein another minimum energy anchor location is chosen such that the previously determined waveform peak or trough in each selected portion coincide when the input segments are overlap-added.
16. A digital waveform concatenation system for use in an acoustic processing application, the system comprising:
a digital waveform provider that produces an input sequence of at least two digital waveform segments, each waveform segment being a sequence of samples; and
a waveform concatenator that:
i. synchronizes successive waveform segments to form a sequence of partially overlapping waveform segments, the overlapping portion of each waveform segment including an optimization zone near a waveform segment boundary, and
ii. weights, and adds selected portions of the input segments to concatenate the input segments so as to produce a single digital waveform;
wherein for segments of voiced speech, the synchronizing includes aligning a largest waveform peak or trough in the optimization zone of each input waveform segment with a corresponding largest waveform peak or trough in an optimization zone of an adjacent waveform segment.
17. A concatenation system according to claim 16 , wherein the acoustic processing application includes a text-to-speech application.
18. A concatenation system according to claim 16 , wherein the acoustic processing application includes a speech broadcast application.
19. A concatenation system according to claim 16 , wherein the acoustic processing application includes a carrier-slot application.
20. A concatenation system according to claim 16 , wherein the waveform segments include at least one of speech diphones and speech triphones.
21. A concatenation system according to claim 16 , wherein the waveform segments include at least one of speech phones and speech demi-phones.
22. A concatenation system according to claim 16 , wherein the waveform segments include at least one of speech demi-syllables, speech syllables, words, and phrases.
23. A concatenation system according to claim 16 , wherein the input segments are filtered before aligning.
24. A digital waveform concatenation system for use in an acoustic processing application, the system comprising:
a digital waveform provider that produces an input sequence of at least two digital waveform segments, each waveform segment being a sequence of samples; and
a waveform concatenator that:
i. synchronizes successive waveform segments to form a sequence of partially overlapping waveform segments, and
ii. weights and adds selected portions of the overlapping waveform segments to concatenate the input waveform segments so as to produce a single digital waveform;
wherein for segments of voiced speech, the synchronizing includes aligning synchronization peaks or troughs in selected portion of each input waveform segment with synchronization peaks or troughs in a corresponding selected portion of an adjacent waveform segment, the location of the selected portions being determined by searching in a neighborhood of waveform segment boundaries for a location where the sum of the weighted energy of the selected portions is minimal.
25. A concatenation system according to claim 24 , wherein the acoustic processing application includes a text-to-speech application.
26. A concatenation system according to claim 24 , wherein the acoustic processing application includes a speech broadcast application.
27. A concatenation system according to claim 24 , wherein the acoustic processing application includes a carrier-slot application.
28. A concatenation system according to claim 24 , wherein the acoustic processing application includes a time-scale modification system.
29. A concatenation system according to claim 24 , wherein the waveform segments include at least one of speech diphones and speech triphones.
30. A concatenation system according to claim 24 , wherein the waveform segments include at least one of speech phones and speech demi-phones.
31. A concatenation system according to claim 24 , wherein the waveform segments include at least one of speech demi-syllables, speech syllables, words, and phrases.
32. A concatenation system according to claim 24 , wherein determining a minimum weighted energy anchor includes using a sliding weighted energy calculation algorithm.
33. A concatenation system according to claim 24 , wherein the input segments are filtered before synchronizing.
34. A concatenation system according to claim 24 , wherein aligning synchronization peaks or troughs includes determining a largest waveform peak or trough in the close neighborhood of each anchor.
35. A concatenation system according to claim 34 , wherein the close neighborhood is an interval of at least one pitch period containing the minimum energy anchor.
36. A concatenation system according to claim 34 , wherein the close neighborhood is the selected portion of the input segment.
37. A concatenation system according to claim 34 , wherein the location of one anchor is chosen such that the synchronization peaks or troughs in each selected portion coincide when the input segments are overlap-added.
38. A digital waveform concatenation system for use in an acoustic processing application, the system comprising:
a digital waveform provider that produces an input sequence of at least two digital waveform segments, each waveform segment being a sequence of samples; and
a waveform concatenator that:
i. synchronizes successive waveform segments to form a sequence of partially overlapping waveform segments, and
ii. weights, and adds selected portions of the overlapping waveform segments to concatenate the input waveform segments so as to produce a single digital waveform;
wherein for pairs of overlapping segments of voiced speech, a first selected portion includes a minimum energy anchor in a location optimized based on determining minimum weighted energy in a neighborhood of the waveform segment boundaries, and a second selected portion is determined by aligning synchronization peaks or troughs in the neighborhood of the waveform segment boundaries.
39. A concatenation system according to claim 38 , wherein the acoustic processing application includes a text-to-speech application.
40. A concatenation system according to claim 38 , wherein the acoustic processing application includes a speech broadcast application.
41. A concatenation system according to claim 38 , wherein the acoustic processing application includes a carrier-slot application.
42. A concatenation system according to claim 38 , wherein the acoustic processing application includes a time-scale modification system.
43. A concatenation system according to claim 38 , wherein the waveform segments include at least one of speech diphones and speech triphones.
44. A concatenation system according to claim 38 , wherein the waveform segments include at least one of speech phones and speech demi-phones.
45. A concatenation system according to claim 38 , wherein the waveform segments include at least one of speech demi-syllables, speech syllables, words, and phrases.
46. A concatenation system according to claim 38 , wherein determining a minimum weighted energy anchor includes using a sliding weighted energy calculation algorithm.
47. A concatenation system according to claim 38 , wherein the input segments are filtered before synchronizing.
48. A concatenation system according to claim 38 , wherein aligning synchronization peaks or troughs includes determining a largest waveform peak or trough in the close neighborhood of the anchor and determining a corresponding peak or trough in the selected portion of the other input segment.
49. A concatenation system according to claim 48 , wherein the close neighborhood is an interval of at least one pitch period containing the minimum weighted energy anchor.
50. A concatenation system according to claim 48 , wherein the close neighborhood is the selected portion of the input segment.