Method of updating sentence generation model and sentence generating apparatus

1. A processor implemented method, comprising:

generating, using a first decoding model, a target sentence corresponding to a source sentence;

generating, using a second decoding model, words included in another target sentence in an order different from a generated word order of the words as also included in the target sentence;

generating, using the second decoding model, reward information associated with the target sentence and the words; and

training, based on the reward information, an updated sentence generation model, including resetting respective weights of nodes in the first decoding model,

wherein the first decoding model, second decoding model, and sentence generation model are machine learning models.

2. The method of claim 1 , wherein the generating of the reward information comprises generating the reward information based on a probability that each of plural words included in the target sentence is generated from the second decoding model.

3. The method of claim 2 , wherein the generating of the reward information further comprises

generating, based on the source sentence and a first word output from the second decoding model at a previous time, a probability that a second word is generated from the second decoding model at a current time.

4. The method of claim 2 , wherein the generating of the reward information further comprises generating the reward information based on a word sequence in which the plural words included in the target sentence are arranged in an order different from an order of the plural words in the target sentence.

5. The method of claim 1 , wherein

the first decoding model is a forward decoding model comprising a recurrent neural network (RNN), and

the second decoding model is a backward decoding model comprising another RNN.

6. The method of claim 1 , wherein the resetting of the respective weights comprises

generating, using the first decoding model, policy information associated with the target sentence, and

resetting the respective weights based on the respective weights corresponding to a specified condition, using the generated policy information and the generated reward information.

7. The method of claim 6 , wherein the generating of the policy information comprises generating the policy information based on a probability that each of plural words included in the target sentence is generated from the first decoding model.

8. The method of claim 7 , wherein the generating of the policy information further comprises generating, based on the source sentence and a first word output from the first decoding model at a previous time, a probability that a second word is generated from the first decoding model at a current time.

9. The method of claim 6 , wherein

the resetting of the respective weights further comprises resetting weights that maximize an objective function defined by reward information and policy information associated with each of target sentences, and

the target sentences are generated from the source sentence using the first decoding model.

10. The method of claim 6 , wherein

the resetting of the respective weights further comprises resetting weights that minimize a loss function defined by an objective function and a probability that a predetermined correct sentence for the source sentence is generated from the first decoding model, and

target sentences are generated from the source sentence using the first decoding model, and the objective function is defined by reward information and policy information associated with each of the target sentences.

11. The method of claim 1 , wherein the generating of the target sentence comprises generating target sentences in response to a final result value, among final result values output from the first decoding model in response to the source sentence, being in a specified range.

12. The method of claim 1 , wherein

the generating of the target sentence comprises generating target sentences based on an output value output from the first decoding model at a first time in response to a word in the source sentence, and

the generating of the reward information comprises generating, using the second decoding model, reward information associated with the target sentences generated at the first time.

13. The method of claim 1 , wherein

the generating of the target sentence comprises generating a specified number of target sentences by sampling an output value output from the first decoding model at a first time in response to a word in the source sentence, and

the generating of the reward information comprises generating, using the second decoding model, reward information associated with the specified number of target sentences.

14. The method of claim 1 , wherein the generating of the target sentence comprises:

generating first target sentences based on an output value output from the first decoding model in response to each of plural words included in the source sentence; and

generating second target sentences in response to a final result value, among final result values output from the first decoding model in response to an entirety of the source sentence, being in a specified range.

15. The method of claim 14 , wherein the generating of the reward information comprises:

generating, using the second decoding model, first reward information associated with the first target sentences; and

generating, using the second decoding model, second reward information associated with the second target sentences.

16. The method of claim 1 , further comprising:

generating, using the second decoding model, the another target sentence corresponding to the source sentence;

generating, using the first decoding model, reward information associated with the another target sentence; and

resetting a weight of each of nodes in the second decoding model based on the generated reward information.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

18. A sentence generating apparatus, comprising:

one or more processors configured to:

generate, using a first decoding model, a target sentence corresponding to a source sentence;

generate, using a second decoding model, words included in another target sentence in an order different from a generated word order of the words as also included in the target sentence;

generate, using the second decoding model, reward information associated with the target sentence and the words; and

train, based on the generated reward information, an updated sentence generation model, including resetting respective weights of nodes in the first decoding model,

wherein the first decoding model, second decoding model, and sentence generation model are machine learning models.

19. The sentence generating apparatus of claim 18 , wherein the one or more processors is further configured to generate the reward information based on a word sequence in which plural words included in the target sentence are arranged in an order different from an order of the plural words in the target sentence.

20. The sentence generating apparatus of claim 19 , wherein the one or more processors is further configured to calculate the reward information by generating, based on the source sentence and a first word output from the second decoding model at a previous time, a probability that a second word is generated from the second decoding model at a current time.

21. The sentence generating apparatus of claim 18 , wherein the one or more processors is further configured to generate the reward information by inputting, to the second decoding model, a word sequence in which characters included in each of plural words included in the target sentence are separated.

22. The sentence generating apparatus of claim 21 , wherein the one or more processors is further configured to calculate, based on the source sentence and a first character output from the second decoding model at a previous time, the reward information by generating a probability that a second character is generated from the second decoding model at a current time.

23. The sentence generating apparatus of claim 21 , wherein the one or more processors is further configured to generate target sentences in response to a final result value, among final result values output from the first decoding model in response to the source sentence, being in a specified range.

24. The sentence generating apparatus of claim 21 , wherein the one or more processors is further configured to:

generate target sentences based on an output value output from the first decoding model at a first time in response to a word in the source sentence; and

generate reward information associated with the target sentences generated at the first time using the second decoding model.

25. The sentence generating apparatus of claim 21 , wherein the one or more processors is further configured to:

generate first target sentences based on an output value output from the first decoding model in response to each of plural words in the source sentence; and

generate second target sentences in response to a final result value, among final result values output from the first decoding model in response to an entirety of the source sentence, being in a specified range.

26. The sentence generating apparatus of claim 25 , wherein the one or more processors is further configured to:

generate, using the second decoding model, first reward information associated with the first target sentences; and

generate, using the second decoding model, second reward information associated with the second target sentences.

27. The sentence generating apparatus of claim 21 , wherein the one or more processors is further configured to:

generate, using the second decoding model, the another target sentence corresponding to the source sentence;

generate, using the first decoding model, reward information associated with the another target sentence; and

reset respective weights of nodes in the second decoding model based on the generated reward information.