Optimizing machine translations for user engagement
Exemplary embodiments relate to techniques for improving a machine translation system. The machine translation system may include one or more models for generating a translation. The system may generate multiple candidate translations, and may present the candidate translations to different groups of users, such as users of a social network. User engagement with the different candidate translations may be measured, and the system may determine which of the candidate translations was most favored by the users. For example, in the context of a social network, the number of times that the translation is liked or shared, or the number of comments associated with the translation, may be used to determine user engagement with the translation. The models of the machine translation system may be modified to favor the most-favored candidate translation. The translation system may repeat this process to continue to tune the models in a feedback loop.
1. A method comprising:
generating a first candidate translation and a second candidate translation using a machine translation system, the first candidate translation being a different translation than the second candidate translation and the machine translation system applying a parameter that determines whether the first candidate translation or the second candidate translation is a more likely translation;
causing the first candidate translation to be provided to a first group of users;
causing the second candidate translation to be provided to a second group of users different from the first group;
receiving one or more indications of engagement with the first candidate translation and the second candidate translation;
determining, based on the one or more indications of engagement, that the first candidate translation was favored over the second candidate translation; and
modifying the machine translation system to favor candidate translations with higher engagement scores by adjusting the parameter in favor of the first candidate translation.
2. The method of claim 1 , further comprising repeating the method in a feedback loop.
3. The method of claim 1 , wherein engagement with the first translation and engagement with the second translation are measured based on at least one of: a number of times that the respective translations are liked on a social network, a number of times that the respective translations are shared on the social network, a number of comments written with respect to the respective translations, or a number of times that a link associated with the respective translation is followed.
4. The method of claim 1 , the determining comprising accessing a first engagement score for the first candidate translation based on the one or more indications of engagement, accessing a second engagement score for the second candidate translation based on the one or more indications of engagement, and comparing the first engagement score to the second engagement score.
5. The method of claim 1 , wherein the machine translation system comprises a translation model that generates one or more destination language hypotheses for source language input material based on a translation model parameter, and the modifying comprises modifying the translation model parameter.
6. The method of claim 1 , wherein the machine translation system comprises a language model that selects between two or more destination language hypotheses for source language input material based on a language model parameter, and the modifying comprises modifying the language model parameter.
7. The method of claim 1 , wherein the first group of users and the second group of users are users of a social network.
8. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
generate a first candidate translation and a second candidate translation using a machine translation system, the first candidate translation being a different translation than the second candidate translation and the machine translation system applying a parameter that determines whether the first candidate translation or the second candidate translation is a more likely translation;
cause the first candidate translation to be provided to a first group of users;
cause the second candidate translation to be provided to a second group of users different from the first group;
receive one or more indications of engagement with the first candidate translation and the second candidate translation;
determine, based on the one or more indications of engagement, that the first candidate translation was favored over the second candidate translation; and
modify the machine translation system to favor candidate translations with higher engagement scores by adjusting the parameter in favor of the first candidate translation.
9. The medium of claim 8 , further storing instructions for causing the processor to repeat the generating through modifying actions in a feedback loop.
10. The medium of claim 8 , wherein engagement with the first translation and engagement with the second translation are measured based on at least one of: a number of times that the respective translations are liked on a social network, a number of times that the respective translations are shared on the social network, a number of comments written with respect to the respective translations, or a number of times that a link associated with the respective translation is followed.
11. The medium of claim 8 , the determining comprising accessing a first engagement score for the first candidate translation based on the one or more indications of engagement, accessing a second engagement score for the second candidate translation based on the one or more indications of engagement, and comparing the first engagement score to the second engagement score.
12. The medium of claim 8 , wherein the machine translation system comprises a translation model that generates one or more destination language hypotheses for source language input material based on a translation model parameter, and the modifying comprises modifying the translation model parameter.
13. The medium of claim 8 , wherein the machine translation system comprises a language model that selects between two or more destination language hypotheses for source language input material based on a language model parameter, and the modifying comprises modifying the language model parameter.
14. The medium of claim 8 , wherein the first group of users and the second group of users are users of a social network.
15. An apparatus comprising:
a non-transitory computer-readable medium configured to store one or more models used by a machine translation system; and
a processor configured to:
generate a first candidate translation and a second candidate translation using the machine translation system, the first candidate translation being a different translation than the second candidate translation and the machine translation system applying a parameter that determines whether the first candidate translation or the second candidate translation is a more likely translation;
cause the first candidate translation to be provided to a first group of users;
cause the second candidate translation to be provided to a second group of users different from the first group;
receive one or more indications of engagement with the first candidate translation and the second candidate translation;
determine, based on the one or more indications of engagement, that the first candidate translation was favored over the second candidate translation; and
modify the machine translation system to favor candidate translations with higher engagement scores by adjusting the parameter in favor of the first candidate translation.
16. The apparatus of claim 15 , wherein the processor is further configured to repeat the generating through modifying actions in a feedback loop.
17. The apparatus of claim 15 , wherein engagement with the first translation and engagement with the second translation are measured based on at least one of: a number of times that the respective translations are liked on a social network, a number of times that the respective translations are shared on the social network, a number of comments written with respect to the respective translations, or a number of times that a link associated with the respective translation is followed.
18. The apparatus of claim 15 , the determining comprising accessing a first engagement score for the first candidate translation based on the one or more indications of engagement, accessing a second engagement score for the second candidate translation based on the one or more indications of engagement, and comparing the first engagement score to the second engagement score.
19. The apparatus of claim 15 , wherein the machine translation system comprises a translation model that generates one or more destination language hypotheses for source language input material based on a translation model parameter, and the modifying comprises modifying the translation model parameter.
20. The apparatus of claim 15 , wherein the machine translation system comprises a language model that selects between two or more destination language hypotheses for source language input material based on a language model parameter, and the modifying comprises modifying the language model parameter.