Artificial intelligence (AI)-based voice sampling apparatus and method for providing speech style in heterogeneous label
Disclosed is an artificial intelligence (AI)-based voice sampling apparatus for providing a speech style in a heterogeneous label, including a rhyme encoder configured to receive a user's voice, extract a voice sample, and analyze a vocal feature included in the voice sample, a text encoder configured to receive text for reflecting the vocal feature, a processor configured to classify the voice sample input to the rhythm encoder into a label according to the vocal feature, provide a weight by measuring a distance between a voice sample corresponding to the label and a voice sample corresponding to a heterogeneous label as a label other than the label and provide a weight by measuring similarity between the label and the heterogeneous label, extract an embedding vector representing the vocal feature, generate a speech style from the embedding vector, and apply the generated speech style to the text, and a rhyme decoder configured to output synthesized voice data in which the speech style is applied to the text by the processor.
1. An artificial intelligence (AI)-based voice sampling apparatus for providing a speech style in a heterogeneous label, the apparatus comprising:
a rhyme encoder configured to receive a user's voice, extract a voice sample, and analyze a vocal feature included in the voice sample;
a text encoder configured to receive an input of text for reflecting the vocal feature;
a processor configured to:
classify the voice sample input to the rhythm encoder into a label according to the vocal feature,
provide a weight by measuring a distance between a voice sample corresponding to the label and a voice sample corresponding to a heterogeneous label as a label other than the label or provide a weight by measuring a similarity between the label and the heterogeneous label,
extract an embedding vector representing the vocal feature,
generate a speech style from the embedding vector, and
apply the generated speech style to the text; and
a rhyme decoder configured to output synthesized voice data in which the speech style is applied to the text by the processor.
2. The apparatus of claim 1 , wherein
the rhyme encoder divides the voice sample by a predetermined label and extracts an embedding vector for the label.
3. The apparatus of claim 1 , wherein
the rhyme encoder extracts the embedding vector through a. vocal feature including at least one of a speech rate, a pronunciation intonation, a pause interval, a pitch, or an intonation of the user included in the voice sample.
4. The apparatus of claim 2 , wherein
the processor separates the voice sample corresponding to the label and the voice sample corresponding to the heterogeneous label and provides a weight by measuring a distance between the separated voice samples.
5. The apparatus of claim 4 , wherein
the processor reduces the weight in proportion to the distance between the separated voice samples.
6. The apparatus of claim 2 , wherein
the processor measures a similarity of each of the label and the heterogeneous label and determines a heterogeneous label having a highest similarity as a similar label of the label.
7. The apparatus of claim 2 , wherein
the processor selects a voice sample closest to a mean value for each vector component of all voice samples in the label, from among a plurality of voice samples input from the user.
8. The apparatus of claim 2 , wherein
the processor selects a voice sample having the smallest sum of distances of the vector components to each of voice samples among a plurality of voice samples input from the user.
9. The apparatus of claim 2 , wherein
the processor selects a voice sample having the smallest sum of distances of the vector components to each of the voice samples in the label among a plurality of voice samples input from the user.
10. An artificial intelligence (AI)-based voice sampling method for providing a speech style in a heterogeneous label, the method comprising:
a first step of receiving a voice sample of a user through a rhyme encoder and analyzing a vocal feature included in the voice sample of the user;
a second step of receiving a text for reflecting the vocal feature through a text encoder;
a third step of extracting an embedding vector from the voice sample, generating a speech style by a processor, and applying the speech style to the text; and
a fourth step of outputting synthesized voice data to which the speech style is applied, through a rhyme decoder,
wherein the third step comprises:
classifying the voice sample input to the rhythm encoder into a label according the vocal feature; and
providing a weight by measuring: a distance between a voice sample corresponding to the label and a voice sample corresponding to a heterogeneous label as a label other than the label, or providing a weight by measuring a similarity between the label and the heterogeneous label.
11. The method of claim 10 , wherein the first step comprises:
receiving a voice sample from the user; and
dividing the voice sample into a preset label to extract an embedding vector for the label.
12. The method of claim 11 , wherein
the embedding vector is extracted through a vocal feature including at least one of a speech rate, a pronunciation intonation, a pause interval, a pitch, or an intonation of the user included in the voice sample.
13. The method of claim 11 , wherein
the embedding vector is extracted through spectral information, sliding information, or a mean value.
14. The method of claim 10 , wherein
the processor selects a voice sample closest to a mean value for each vector component of all voice samples in the label, from among a plurality of voice samples input from the user.
15. The method of claim 10 , wherein
the processor selects a voice sample having the smallest sum of distances of the vector components to each of voice samples among a plurality of voice samples input from the user.
16. The method of claim 10 , wherein
the processor selects a voice sample having the smallest sum of distances of the vector components to each of the voice samples in the label among a plurality of voice samples input from the user.
17. The method of claim 10 , wherein
the receiving of the voice sample comprises receiving the voice sample from the user in real time from the user within a predetermined time interval.
18. An artificial intelligence (AI)-based voice sampling method for providing a speech style in a heterogeneous label, the method comprising:
receiving a voice sample of a user through a rhyme encoder and analyzing a vocal feature included in the voice sample of the user;
receiving a text for reflecting the vocal feature through a text encoder;
extracting an embedding vector from the voice sample, generating a speech style by a processor, and applying the speech style to the text; and
outputting synthesized voice data to which the speech style is applied, through a rhyme decoder,
wherein the processor selects a voice sample having the smallest sum of distances of the vector components to each of voice samples among a plurality of voice samples input from the user.