Speech speed adjustment method and apparatus, electronic device, and readable storage medium

1 . A method for speech speed adjustment, comprising:

acquiring a text to be synthesized;

inputting the text to be synthesized to a speech synthesis model, and acquiring a spectrum corresponding to the text to be synthesized output by the speech synthesis model, wherein the speech synthesis model comprises an encoding network, an attention network and a decoding network, with the encoding network being used for converting the input text to be synthesized into an acoustic feature sequence, the attention network being used for outputting the attention vector, and the decoding network being used for outputting the spectrum corresponding to the text to be synthesized according to the attention vector being input, the acoustic feature sequence and a state transition control factor, the spectrum corresponding to the text to be synthesized comprises a plurality of first spectrums, and the state transition control factor is used for controlling a number of the plurality of first spectrums; and

acquiring an audio of the spectrum corresponding to the text to be synthesized according to the spectrum corresponding to the text to be synthesized, wherein the audio has a specified speech speed;

wherein when the decoding network predicts the spectrum, performing a weighted calculation on the attention vector and the acoustic feature sequence to obtain a vector of a current step; acquiring an initial state quantity of the current step based on the vector of the current step; generating a mask according to the state transition control factor, and performing weighted fusion on the initial state quantity of the current step and a state quantity of the previous step according to the mask to obtain a state quantity of the current step.

2 . The method of claim 1 , wherein when the state transition control factor is smaller than a preset threshold, the specified speech speed of the audio is smaller than a reference speech speed;

when the state transition control factor is greater than the preset threshold, the specified speech speed of the audio is greater than the reference speech speed; and

when the state transition control factor is equal to the preset threshold, the specified speech speed of the audio is equal to the reference speech speed.

3 . The method of claim 1 , wherein the decoding network comprises a first fully connected layer, a second fully connected layer, a linear layer, and a recurrent neural network layer;

the decoding network being used for outputting the spectrum corresponding to the text to be synthesized according to the attention vector being input, the acoustic feature sequence and the state transition control factor, comprises:

performing the weighted calculation on the attention vector and the acoustic feature sequence to obtain the vector of the current step, and inputting the vector of the current step to the recurrent neural network layer;

the recurrent neural network layer acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor, and a state quantity of the previous step;

inputting the state quantity of the current step to the first fully connected layer, and acquiring a first spectrum of the current step output by the first fully connected layer;

inputting the state quantity of the current step to the second fully connected layer, and acquiring a stop token output by the second fully connected layer;

when the stop token indicates that an end position of the text to be synthesized is not reached, extracting the first spectrum of the current step through the linear layer and inputting the first spectrum of the current step to the attention network, so that the attention network updates the attention vector; and

returning to perform the weighted calculation on the attention vector and the acoustic feature sequence to obtain the vector of the current step, and inputting the vector of the current step to the recurrent neural network layer, with the recurrent neural network layer acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor and the state quantity of the previous step; inputting the state quantity of the current step to the first fully connected layer, and acquiring the first spectrum of the current step output by the first fully connected layer; inputting the state quantity of the current step to the second fully connected layer, and acquiring a stop token output by the second fully connected layer until the stop token indicates that the end position of the text to be synthesized has been reached.

4 . The method of claim 3 , wherein before the acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor, and the state quantity of the previous step, the method further comprises:

updating the size of the state transition control factor corresponding to the current step.

5 . The method of claim 4 , wherein the updating the size of the state transition control factor corresponding to the current step, comprises:

updating the size of the state transition control factor corresponding to the current step according to one or more selected from the group consisting of the specified speech speed, the acoustic feature sequence corresponding to the text to be synthesized, and the importance of the text content corresponding to the current step.

6 . The method of claim 3 , wherein the inputting the state quantity of the current step to the second fully connected layer and acquiring the stop token output by the second fully connected layer, comprises:

inputting the state quantity of the current step to the second fully connected layer, and performing a weighted calculation on the state quantity of the current step through the second fully connected layer to acquire a weighted calculation result as the stop token; and

after acquiring the weighted calculation result and as the stop token, further comprises:

when the stop token is greater than or equal to a preset threshold, determining that the stop token has reached the end position of the text to be synthesized;

when the stop token is smaller than the preset threshold, determining that the stop token has not reached the end position of the text to be synthesized; or,

the inputting the state quantity of the current step to the second fully connected layer and acquiring the stop token output by the second fully connected layer, comprises:

inputting the state quantity of the current step to the second fully connected layer, performing a weighted calculation on the state quantity of the current step through the second fully connected layer, and classifying the weighted calculation result by using a sigmoid function to acquire the classification result as a stop token.

7 . The method of claim 3 , wherein the extracting the first spectrum of the current step through the linear layer and inputting it to the attention network, so that the attention network updates the attention vector, comprises:

inputting the effective information extracted from the existing first spectrums to the attention network through the linear layer, so that the attention network generates and updates the attention vector according to the effective information extracted from the existing first spectrums; or

inputting the effective information extracted from the existing first spectrums to the recurrent neural network layer through the linear layer, and inputting the effective information extracted from the existing first spectrums to the attention network through the recurrent neural network layer, so that the attention network generates and updates the attention vector according to the effective information extracted from the existing first spectrums.

8 . The method of claim 1 , wherein before the acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor and the state quantity of the previous step, the method further comprises:

updating the size of the state transition control factor corresponding to the current step.

9 . The method of claim 1 , wherein before the acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor and the state quantity of the previous step, the method further comprises:

updating the size of the state transition control factor corresponding to the current step.

10 . An electronic device comprising:

a memory and at least one processor,

wherein the memory is configured to store computer program instructions; and

the at least one processor is configured to execute the computer program instructions, causing the electronic device to perform a speech speed adjustment method, and the speech speed adjustment method comprises:

acquiring a text to be synthesized;

inputting the text to be synthesized to a speech synthesis model, and acquiring a spectrum corresponding to the text to be synthesized output by the speech synthesis model, wherein the speech synthesis model comprises an encoding network, an attention network and a decoding network, with the encoding network being used for converting the input text to be synthesized into an acoustic feature sequence, the attention network being used for outputting the attention vector, and the decoding network being used for outputting the spectrum corresponding to the text to be synthesized according to the attention vector being input, the acoustic feature sequence and a state transition control factor, the spectrum corresponding to the text to be synthesized comprises a plurality of first spectrums, and the state transition control factor is used for controlling a number of the plurality of first spectrums; and

acquiring an audio of the spectrum corresponding to the text to be synthesized according to the spectrum corresponding to the text to be synthesized, wherein the audio has a specified speech speed;

when the decoding network predicts the spectrum, performing a weighted calculation on the attention vector and the acoustic feature sequence to obtain a vector of a current step; acquiring an initial state quantity of the current step based on the vector of the current step; generating a mask according to the state transition control factor, and performing weighted fusion on the initial state quantity of the current step and a state quantity of the previous step according to the mask to obtain a state quantity of the current step.

11 . The electronic device of claim 10 , wherein when the state transition control factor is smaller than a preset threshold, the specified speech speed of the audio is smaller than a reference speech speed;

when the state transition control factor is greater than the preset threshold, the specified speech speed of the audio is greater than the reference speech speed; and

when the state transition control factor is equal to the preset threshold, the specified speech speed of the audio is equal to the reference speech speed.

12 . The electronic device of claim 10 , wherein the decoding network comprises a first fully connected layer, a second fully connected layer, a linear layer and a recurrent neural network layer;

the decoding network being used for outputting the spectrum corresponding to the text to be synthesized according to the attention vector being input, the acoustic feature sequence and the state transition control factor, comprises:

performing the weighted calculation on the attention vector and the acoustic feature sequence to obtain the vector of the current step, and inputting the vector of the current step to the recurrent neural network layer;

the recurrent neural network layer acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor, and a state quantity of the previous step;

inputting the state quantity of the current step to the first fully connected layer, and acquiring a first spectrum of the current step output by the first fully connected layer; inputting the state quantity of the current step to the second fully connected layer, and acquiring a stop token output by the second fully connected layer;

when the stop token indicates that an end position of the text to be synthesized is not reached, extracting the first spectrum of the current step through the linear layer and inputting the first spectrum of the current step to the attention network, so that the attention network updates the attention vector; and

returning to perform the weighted calculation on the attention vector and the acoustic feature sequence to obtain the vector of the current step, and inputting the vector of the current step to the recurrent neural network layer, with the recurrent neural network layer acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor and the state quantity of the previous step; inputting the state quantity of the current step to the first fully connected layer, and acquiring the first spectrum of the current step output by the first fully connected layer; inputting the state quantity of the current step to the second fully connected layer, and acquiring a stop token output by the second fully connected layer until the stop token indicates that the end position of the text to be synthesized has been reached.

13 . The electronic device of claim 12 , wherein before the acquiring the state quantity of the current step according to the vector of the current step, the state transition control factor and the state quantity of the previous step, the apparatus is further configured to:

update the size of the state transition control factor corresponding to the current step.

14 . The electronic device of claim 13 , wherein the updating the size of the state transition control factor corresponding to the current step, comprises:

updating the size of the state transition control factor corresponding to the current step according to one or more selected from the group consisting of the specified speech speed, the acoustic feature sequence corresponding to the text to be synthesized and the importance of the text content corresponding to the current step.

15 . A non-transitory computer-readable storage medium comprising: computer program instructions,

wherein the computer program instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform a speech speed adjustment method, and the speech speed adjustment method comprises:

acquiring a text to be synthesized;

inputting the text to be synthesized to a speech synthesis model, and acquiring a spectrum corresponding to the text to be synthesized output by the speech synthesis model, wherein the speech synthesis model comprises an encoding network, an attention network and a decoding network, with the encoding network being used for converting the input text to be synthesized into an acoustic feature sequence, the attention network being used for outputting the attention vector, and the decoding network being used for outputting the spectrum corresponding to the text to be synthesized according to the attention vector being input, the acoustic feature sequence and a state transition control factor, the spectrum corresponding to the text to be synthesized comprises a plurality of first spectrums, and the state transition control factor is used for controlling a number of the plurality of first spectrums; and

acquiring an audio of the spectrum corresponding to the text to be synthesized according to the spectrum corresponding to the text to be synthesized, wherein the audio has a specified speech speed;

when the decoding network predicts the spectrum, performing a weighted calculation on the attention vector and the acoustic feature sequence to obtain a vector of a current step; acquiring an initial state quantity of the current step based on the vector of the current step; generating a mask according to the state transition control factor, and performing weighted fusion on the initial state quantity of the current step and a state quantity of the previous step according to the mask to obtain a state quantity of the current step.