Speech synthesis unit selection
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting units for speech synthesis. One of the methods includes determining a sequence of text units that each represent a respective portion of text for speech synthesis; and determining multiple paths of speech units that each represent the sequence of text units by selecting a first speech unit that includes speech synthesis data representing a first text unit; selecting multiple second speech units including speech synthesis data representing a second text unit based on (i) a join cost to concatenate the second speech unit with a first speech unit and (ii) a target cost indicating a degree that the second speech unit corresponds to the second text unit; and defining paths from the selected first speech unit to each of the multiple second speech units to include in the multiple paths of speech units.
1. A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising:
receiving data indicating text for speech synthesis;
determining a sequence of text units that each represent a respective portion of the text;
generating a lattice of candidate speech units comprising a same predetermined L number of speech units for each text unit in the sequence of text units and K number of specific paths that extend through the lattice by, for a last speech unit of each corresponding specific path of the K number of specific paths:
determining X number of speech units to extend from the last speech unit of the corresponding specific path of the K number of specific paths, wherein X corresponds to a value represented by a ratio of L to K; and
adding the determined X number of speech units to the last speech unit of the corresponding specific path of the K number of specific paths; and
providing synthesized speech data according to the speech units of one of the specific paths selected from the K number of specific paths that extend through the generated lattice.
2. The method of claim 1 , wherein determining the sequence of text units that each represent a respective portion of the text comprises determining the sequence of text units that each represent a distinct portion of the text, separate from the portions of text represented by the other text units.
3. The method of claim 1 , wherein providing the synthesized speech data comprises providing the synthesized speech data to a device that causes the device to generate audible data for the text.
4. The method of claim 1 , wherein generating the lattice comprises:
selecting, from the predetermined L number of speech units for a beginning text unit in the sequence of text units within a location at a beginning of the text, two or more beginning speech units that each comprise speech synthesis data representing the beginning text unit; and
extending a specific path of the K number of specific paths through the lattice from each of the two or more beginning speech units.
5. The method of claim 1 , wherein the operations further comprise determining the predetermined L number of speech units for each text unit in the sequence of text units from a speech unit corpus.
6. The method of claim 1 , wherein each added speech unit of the determined X number of speech units added to the last speech unit of the corresponding specific path of the K number of specific paths is based on:
a join cost to concatenate the added speech unit with the last speech unit of the corresponding specific path based on respective acoustic parameters associated with the added speech unit; and
a target cost indicating a degree that the added speech unit corresponds to the text unit to which the added speech unit corresponds in the lattice.
7. The method of claim 6 , wherein the respective acoustic parameters comprise a speech unit context indicating at least one of an adjacent speech unit that occurred before the added speech unit when a waveform associated with the added speech unit was created or an adjacent speech unit that occurred after the added speech unit when the waveform associated with the added speech unit was created.
8. The method of claim 1 , wherein generating the lattice comprises sequentially populating the lattice with speech units for the respective text units, and continuing no more than K number of specific paths for each of the text units.
9. The method of claim 1 , wherein generating the lattice comprises sequentially selecting sets of speech units for the respective text units in the sequence of text units, wherein selecting the set of speech units for a text unit comprises selecting, for the position in the lattice corresponding to the text unit, one or more of the K number of paths to branch into multiple paths and one or more of the multiple paths to prune.
10. The method of claim 1 , wherein generating the lattice comprises determining a subset of the specific paths to branch into multiple paths based on a total cost that includes join costs and target costs for a sequence of three or more speech units.
11. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving data indicating text for speech synthesis;
determining a sequence of text units that each represent a respective portion of the text;
generating a lattice of candidate speech units comprising a same predetermined L number of speech units for each text unit in the sequence of text units and K number of specific paths that extend through the lattice by, for a last speech unit of each corresponding specific path of the K number of specific paths:
determining X number of speech units to extend from the last speech unit of the corresponding specific path of the K number of specific paths, wherein X corresponds to a value represented by a ratio of L to K; and
adding the determined X number of speech units to the last speech unit of the corresponding specific path of the K number of specific paths; and
providing synthesized speech data according to the speech units of one of the specific paths selected from the K number of specific paths that extend through the generated lattice.
12. The system of claim 11 , wherein determining the sequence of text units that each represent a respective portion of the text comprises determining the sequence of text units that each represent a distinct portion of the text, separate from the portions of text represented by the other text units.
13. The system of claim 11 , wherein providing the synthesized speech data comprises providing the synthesized speech data to a device that causes the device to generate audible data for the text.
14. The system of claim 11 , wherein generating the lattice comprises:
selecting, from the predetermined L number of speech units for a beginning text unit in the sequence of text units within a location at a beginning of the text, two or more beginning speech units that each comprise speech synthesis data representing the beginning text unit; and
extending a specific path of the K number of specific paths through the lattice from each of the two or more beginning speech units.
15. The system of claim 11 , wherein the operations further comprise determining the predetermined L number of speech units for each text unit in the sequence of text units from a speech unit corpus.
16. The system of claim 11 , wherein each added speech unit of the determined X number of speech units added to the last speech unit of the corresponding specific path of the K number of specific paths is based on:
a join cost to concatenate the added speech unit with the last speech unit of the corresponding specific path based on respective acoustic parameters associated with the added speech unit; and
a target cost indicating a degree that the added speech unit corresponds to the text unit to which the added speech unit corresponds in the lattice.
17. The system of claim 16 , wherein the respective acoustic parameters comprise a speech unit context indicating at least one of an adjacent speech unit that occurred before the added speech unit when a waveform associated with the added speech unit was created or an adjacent speech unit that occurred after the added speech unit when the waveform associated with the added speech unit was created.
18. The system of claim 11 , wherein generating the lattice comprises sequentially populating the lattice with speech units for the respective text units, and continuing no more than K number of specific paths for each of the text units.
19. The system of claim 11 , wherein generating the lattice comprises sequentially selecting sets of speech units for the respective text units in the sequence of text units, wherein selecting the set of speech units for a text unit comprises selecting, for the position in the lattice corresponding to the text unit, one or more of the K number of paths to branch into multiple paths and one or more of the multiple paths to prune.
20. The system of claim 11 , wherein generating the lattice comprises determining a subset of the specific paths to branch into multiple paths based on a total cost that includes join costs and target costs for a sequence of three or more speech units.