IP Library Granted Patent US 9,830,904
Granted Patent B2
US 9,830,904 · App. 15/185,259 · Granted Nov 28, 2017

Text-to-speech device, text-to-speech method, and computer program product

Inventors: Yu Nasu (Tokyo, JP); Masatsune Tamura (Kanagawa, JP); Ryo Morinaka (Tokyo, JP); Masahiro Morita (Kanagawa, JP)
Assignee: KABUSHIKI KAISHA TOSHIBA
G10L13/10G10L13/033G10L13/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,830,904
App. No.
15/185,259
Granted
Nov 28, 2017
Kind
B2
Abstract

According to an embodiment, a text-to-speech device includes a context acquirer, an acoustic model parameter acquirer, a conversion parameter acquirer, a converter, and a waveform generator. The context acquirer is configured to acquire a context sequence affecting fluctuations in voice. The acoustic model parameter acquirer is configured to acquire an acoustic model parameter sequence that corresponds to the context sequence and represents an acoustic model in a standard speaking style of a target speaker. The conversion parameter acquirer is configured to acquire a conversion parameter sequence corresponding to the context sequence to convert an acoustic model parameter in the standard speaking style into one in a different speaking style. The converter is configured to convert the acoustic model parameter sequence using the conversion parameter sequence. The waveform generator is configured to generate a voice signal based on the acoustic model parameter sequence acquired after conversion.

Claims (59)

1. A text-to-speech device comprising:

one or more processors configured to:

acquire a context sequence that is an information sequence affecting fluctuations in voice;

acquire an acoustic model parameter sequence corresponding to the context sequence, the acoustic model parameter sequence representing a standard speaking style of a target speaker;

acquire a conversion parameter sequence corresponding to the context sequence, the conversion parameter sequence being used in converting an acoustic model parameter in the standard speaking style into one in a speaking style different from the standard speaking style;

convert the acoustic model parameter sequence using the conversion parameter sequence; and

generate a voice signal based on the acoustic model parameter sequence acquired after conversion.

2. The device according to claim 1 , wherein the context sequence includes at least a phoneme sequence.

3. The device according to claim 1 , further comprising:

an acoustic model parameter storage configured to store a plurality of acoustic model parameters classified according to contexts and store first classification information used in determining one of the acoustic model parameters corresponding to a given context; and

a conversion parameter storage configured to store a plurality of conversion parameters classified according to contexts and store second classification information used in determining one of the conversion parameters corresponding to a given context,

wherein the one or more processors is configured to:

determine, based on the first classification information stored in the acoustic model parameter storage, the acoustic model parameter sequence corresponding to the acquired context sequence, and

determine, based on the second classification information stored in the conversion parameter storage, the conversion parameter sequence corresponding to the acquired context sequence.

4. The device according to claim 3 , wherein the conversion parameter is created using voice samples uttered by a certain speaker in a standard speaking style and voice samples uttered by the same speaker in a different speaking style from the standard speaking style.

5. The device according to claim 3 , wherein

the acoustic model parameter is created using voice samples uttered by the target speaker, and

the conversion parameter is created using voice samples uttered by a speaker different from the target speaker.

6. The device according to claim 3 , wherein

the acoustic model parameter is created using voice samples uttered by the target speaker in a speaking style expressing neutral feeling, and

the conversion parameter represents information used in converting an acoustic model parameter of the speaking style expressing neutral feeling into one expressing a feeling other than neutral.

7. The device according to claim 1 , wherein

the acoustic model is a probabilistic model in which output probabilities of respective phonetic parameters that represent characteristics of a voice are expressed using Gaussian distribution,

the acoustic model parameter includes a mean vector representing a mean of an output probability distribution of each phonetic parameter,

the conversion parameter represents a vector having the same dimensionality as the mean vector included in the acoustic model parameter, and

the one or more processors is further configured to add a conversion parameter included in the conversion parameter sequence to a mean vector included in the acoustic model parameter sequence to generate a post-conversion acoustic model parameter sequence.

8. The device according to claim 1 , further comprising:

a plurality of conversion parameter storages configured to store conversion parameters corresponding to mutually different speaking styles,

wherein the one or more processors is further configured to:

select one of the plurality of conversion parameter storages, and

acquire the conversion parameter sequence from the selected conversion parameter storage.

9. The device according to claim 1 , further comprising:

a plurality of conversion parameter storages configured to store conversion parameters corresponding to mutually different speaking styles,

wherein the one or more processors is further configured to:

select two or more of the plurality of conversion parameter storages, wherein

acquire the conversion parameter sequence from each of the selected two or more conversion parameter storages, and

convert the acoustic model parameter sequence using the two or more conversion parameter sequences.

10. The device according to claim 9 ,

wherein the one or more processors is further configured to:

control ratios at which the respective conversion parameters acquired from the selected two or more of the conversion parameter storages are to be reflected in the acoustic model parameters.

11. The device according to claim 1 , further comprising:

a plurality of acoustic model parameter storages configured to store the acoustic model parameters corresponding to mutually different speakers,

wherein the one or more processors is further configured to:

select one of the plurality of acoustic model parameter storages, and

acquire the acoustic model parameter sequence from the selected acoustic model parameter storage.

12. The device according to claim 11 ,

wherein the one or more processors is further configured to convert the acoustic model parameter stored in one of the acoustic model parameter storages into the acoustic model parameter corresponding to a specific speaker using speaker adaptation, and write the acoustic model parameter acquired by conversion in the acoustic model parameter storage corresponding to the specific speaker.

13. A text-to-speech method comprising:

acquiring by one or more processors, a context sequence that is an information sequence affecting fluctuations in voice;

acquiring by the one or more processors, an acoustic model parameter sequence corresponding to the context sequence, the acoustic model parameter sequence representing an acoustic model in a standard speaking style of a target speaker;

acquiring by the one or more processors, a conversion parameter sequence corresponding to the context sequence, the conversion parameter sequence being used in converting an acoustic model parameter in the standard speaking style into one in a speaking style different from the standard speaking style;

converting by the one or more processors, the acoustic model parameter sequence using the conversion parameter sequence; and

generating by the one or more processors, a voice signal based on the acoustic model parameter sequence acquired after conversion.

14. A computer program product comprising a non-transitory computer-readable medium containing a program executed by a computer, the program causing the computer to execute:

acquiring a context sequence that is an information sequence affecting fluctuations in voice;

acquiring an acoustic model parameter sequence corresponding to the context sequence, the acoustic model parameter sequence representing an acoustic model in a standard speaking style of a target speaker;

acquiring a conversion parameter sequence corresponding to the context sequence, the conversion parameter sequence being used in converting an acoustic model parameter in the standard speaking style into one in a speaking style different from the standard speaking style;

converting the acoustic model parameter sequence using the conversion parameter sequence; and

generating a voice signal based on the acoustic model parameter sequence acquired after conversion.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S ADDRESS PREVIOUSLY RECORDED ON REEL 048547 FRAME 0187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST. Recorded May 6, 2020
From: KABUSHIKI KAISHA TOSHIBA
To: TOSHIBA DIGITAL SOLUTIONS CORPORATION
Reel/Frame 052595/0307 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ADD SECOND RECEIVING PARTY PREVIOUSLY RECORDED AT REEL: 48547 FRAME: 187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 13, 2019
From: KABUSHIKI KAISHA TOSHIBA
To: KABUSHIKI KAISHA TOSHIBA; TOSHIBA DIGITAL SOLUTIONS CORPORATION
Reel/Frame 050041/0054 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 8, 2019
From: KABUSHIKI KAISHA TOSHIBA
To: TOSHIBA DIGITAL SOLUTIONS CORPORATION
Reel/Frame 048547/0187 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2016
From: NASU, YU; TAMURA, MASATSUNE; MORINAKA, RYO; MORITA, MASAHIRO
To: KABUSHIKI KAISHA TOSHIBA
Reel/Frame 040144/0904 →
Continuity (2)
Continuation PCTJP2013084356 · Dec 20, 2013
Related Publication 20160300564A1 · Oct 13, 2016