Method and apparatus with natural language generation

1. A processor-implemented method of generating a natural language, the method comprising:

generating a word-level latent variable based on an embedding vector of each word of an input utterance;

determining attention information related to the input utterance by applying the generated word-level latent variable to a neural network model;

generating a sentence-level latent variable with respect to plural words of the input utterance based on the embedded vector, and outputting a sampled embedding vector based on a probability distribution of the sentence-level variable; and

outputting a natural language response that corresponds to the input utterance based on the calculated attention information and the sampled embedding vector.

2. The method of claim 1 , wherein the generating of the latent variable from the embedding vector comprises generating the word-level latent variable from an embedding vector of each of plural words in the input utterance.

3. The method of claim 2 , wherein the generating of the latent variable from the embedding vector comprises generating, as the word-level latent variable, a probability distribution by extending the embedding vector of each word in the input utterance to a predetermined range in a predefined vector space.

4. The method of claim 2 , wherein the determining of the attention information related to the input utterance comprises calculating attention information corresponding to each word in the input utterance by transmitting the word-level latent variable to the neural network.

5. The method of claim 2 , wherein the determining of the attention information related to the input utterance comprises calculating attention information corresponding to an entire input utterance by inputting latent variables of a plurality of words included in the input utterance into the neural network.

6. The method of claim 4 , wherein the generating of the latent variable from the embedding vector comprises generating a sentence-level latent variable from an embedding vector corresponding to a complete input utterance.

7. The method of claim 6 , further comprising:

outputting, to a decoding model, an embedding vector sampled based on a predesignated condition from a probability distribution corresponding to the sentence-level latent variable.

8. The method of claim 7 , wherein the outputting of the natural language response comprises outputting the natural language response corresponding to the input utterance based on attention information determined from word-level latent variables and the sampled embedding vector.

9. An apparatus for generating a natural language, the apparatus comprising:

a processor configured to:

generate a word-level latent variable based on an embedding vector of each word of an input utterance;

determine attention information related to the input utterance by applying the generated word-level latent variable to a neural network model,

generate a sentence-level latent variable with respect to plural words of the input utterance based on the embedded vector, and outputting a sampled embedding vector based on a probability distribution of the sentence-level variable; and

output a natural language response that corresponds to the input utterance based on the calculated attention information and the sampled embedding vector.

10. The apparatus of claim 9 , further comprising a memory coupled to the processor, the memory storing instructions that, when executed by the processor, generates the word-level latent variable from the embedding vector that corresponds to the input utterance, calculates the attention information related to the input utterance by applying the generated word-level latent variable to the attention model, and output the natural language response that corresponds to the input utterance based on the calculated attention information.

11. A processor-implemented method of generating a natural language, the method comprising:

generating a sentence-level latent variable from a first embedding vector that corresponds to a complete input utterance;

generating a word-level latent variable based on a second embedding vector of each word included in the input utterance and the sentence-level latent variable;

determining attention information related to the input utterance by applying the generated word-level latent variable to an attention model; and

outputting a natural language response that corresponds to the input utterance based on the calculated attention information.

12. The method of claim 11 , wherein the generating of the word-level latent variable comprises generating the word-level latent variable by comparing the sentence-level latent variable to a probability distribution generated by extending the second embedding vector of each word in the input utterance to a predetermined range in a predefined vector space.

13. The method of claim 11 , wherein the determining of the attention information related to the input utterance comprises calculating attention information corresponding to each word in the input utterance by transmitting the word-level latent variable to the neural network model.

14. The method of claim 13 , further comprising:

outputting, to a decoding model, a third embedding vector sampled based on a predesignated condition from a probability distribution corresponding to the sentence-level latent variable.

15. The method of claim 14 , wherein the outputting of the natural language response comprises outputting the natural language response that corresponds to the input utterance by applying the third embedding vector and the attention information to the decoding model.

16. The method of claim 15 , wherein the outputting of the natural language response comprises outputting the natural language response that corresponds to the input utterance by decoding the third embedding vector in a vector space of a keyword defined by the attention information.

17. An apparatus for generating a natural language, the apparatus comprising:

a processor configured to:

generate a sentence-level latent variable from a first embedding vector corresponding to a complete input utterance;

generate a word-level latent variable based on a second embedding vector of each word in the input utterance and the sentence-level latent variable;

determine attention information related to the input utterance by applying the generated word-level latent variable to a neural network model, and

output a natural language response that corresponds to the input utterance based on the calculated attention information.

18. The apparatus of claim 17 , further comprising a memory coupled to the processor, the memory storing instructions that, when executed by the processor generates a sentence-level latent variable from the first embedding vector that corresponds to the complete input utterance, generates a word-level latent variable based on a second embedding vector of each word in the input utterance and the sentence-level latent variable, calculates attention information related to the input utterance by applying the generated word-level latent variable to an attention model, and outputs a natural language response that corresponds to the input utterance based on the calculated attention information.

19. The apparatus of claim 18 , wherein the first embedding vector indicates a structure of a sentence that corresponds to the input utterance, and the structure indicates a start point of the sentence and an end point of the sentence.