Optimizing machine translations for user engagement
Exemplary embodiments relate to techniques for improving a machine translation system. The machine translation system may include one or more models for generating a translation. The system may generate multiple candidate translations, and may present the candidate translations to different groups of users, such as users of a social network. User engagement with the different candidate translations may be measured, and the system may determine which of the candidate translations was most favored by the users. For example, in the context of a social network, the number of times that the translation is liked or shared, or the number of comments associated with the translation, may be used to determine user engagement with the translation. The models of the machine translation system may be modified to favor the most-favored candidate translation. The translation system may repeat this process to continue to tune the models in a feedback loop.
1. A method comprising:
(a) generating a first candidate translation and a second candidate translation using a machine translation system, the first candidate translation being a different translation than the second candidate translation;
(b) causing the first candidate translation to be provided to a first group of users;
(c) causing the second candidate translation to be provided to a second group of users different from the first group;
(d) receiving one or more indications of engagement with the first candidate translation and the second candidate translation, wherein engagement with the first translation and engagement with the second translation are measure based on at least one of: a number of times that the respective translations are liked on a social network, a number of times that the respective translations are shared on the social network, a number of comments written with respect to the respective translations, or a number of times that a link associated with the respective translation is followed;
(e) determining, based on the one or more indications of engagement, that the first candidate translation was favored over the second candidate translation, the determining comprising computing a first engagement score for the first candidate translation based on the one or more indications of engagement, computing a second engagement score for the second candidate translation based on the one or more indications of engagement, and comparing the first engagement score to the second engagement score; and
(f) modifying the machine translation system in favor of the first candidate translation.
2. The method of claim 1 , wherein operation (b) comprises surfacing the first candidate translation to the first group of users on the social network, and operation (c) comprises surfacing the second candidate translation to the second group of users on the social network.
3. The method of claim 1 , wherein engagement with the first translation and engagement with the second translation are measured by querying the first group of users and the second group of users as to whether the respective translations are understandable or useable.
4. The method of claim 1 , wherein the translation system comprises a translation model that generates one or more destination language hypotheses for source language input material based on a translation model parameter, and operation (f) comprises modifying the translation model parameter.
5. The method of claim 1 , wherein the translation system comprises a language model that selects between two or more destination language hypotheses for source language input material based on a language model parameter, and operation (f) comprises modifying the language model parameter.
6. The method of claim 1 , further comprising repeating operation (a)-(f) using the modified machine translation system to further tune the machine translation system.
7. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
(a) generate a first candidate translation and a second candidate translation using a machine translation system, the first candidate translation being a different translation than the second candidate translation;
(b) cause the first candidate translation to be provided to a first group of users;
(c) cause the second candidate translation to be provided to a second group of users different from the first group;
(d) receive one or more indications of engagement with the first candidate translation and the second candidate translation, wherein engagement with the first translation and engagement with the second translation are measure based on at least one of: a number of times that the respective translations are liked on a social network, a number of times that the respective translations are shared on the social network, a number of comments written with respect to the respective translations, or a number of times that a link associated with the respective translation is followed;
(e) determine, based on the one or more indications of engagement, that the first candidate translation was favored over the second candidate translation, the determining comprising computing a first engagement score for the first candidate translation based on the one or more indications of engagement, computing a second engagement score for the second candidate translation based on the one or more indications of engagement, and comparing the first engagement score to the second engagement score; and
(f) modify the machine translation system in favor of the first candidate translation.
8. The medium of claim 7 , wherein operation (b) comprises surfacing the first candidate translation to the first group of users on the social network, and operation (c) comprises surfacing the second candidate translation to the second group of users on the social network.
9. The medium of claim 7 , wherein engagement with the first translation and engagement with the second translation are measured by querying the first group of users and the second group of users as to whether the respective translations are understandable or useable.
10. The medium of claim 7 , wherein the translation system comprises a translation model that generates one or more destination language hypotheses for source language input material based on a translation model parameter, and operation (f) comprises modifying the translation model parameter.
11. The medium of claim 7 , wherein the translation system comprises a language model that selects between two or more destination language hypotheses for source language input material based on a language model parameter, and operation (f) comprises modifying the language model parameter.
12. The medium of claim 7 , further storing instructions for repeating operations (a)-(f) using the modified machine translation system to further tune the machine translation system.
13. An apparatus comprising:
a non-transitory computer-readable medium configured to store one or more models used by a machine translation system;
a processor configured to:
(a) generate a first candidate translation and a second candidate translation using a machine translation system, the first candidate translation being a different translation than the second candidate translation,
(b) cause the first candidate translation to be provided to a first group of users,
(c) cause the second candidate translation to be provided to a second group of users different from the first group,
(d) receive one or more indications of engagement with the first candidate translation and the second candidate translation, wherein engagement with the first translation and engagement with the second translation are measure based on at least one of: a number of times that the respective translations are liked on a social network, a number of times that the respective translations are shared on the social network, a number of comments written with respect to the respective translations, or a number of times that a link associated with the respective translation is followed,
(e) determine, based on the one or more indications of engagement, that the first candidate translation was favored over the second candidate translation the determining comprising computing a first engagement score for the first candidate translation based on the one or more indications of engagement, computing a second engagement score for the second candidate translation based on the one or more indications of engagement, and comparing the first engagement score to the second engagement score, and
(f) modify the machine translation system in favor of the first candidate translation.
14. The apparatus of claim 13 , wherein operation (b) comprises surfacing the first candidate translation to the first group of users on the social network, and operation (c) comprises surfacing the second candidate translation to the second group of users on the social network.
15. The apparatus of claim 13 , wherein engagement with the first translation and engagement with the second translation are measured by querying the first group of users and the second group of users as to whether the respective translations are understandable or useable.
16. The apparatus of claim 13 , wherein the one or more models comprise a translation model that generates one or more destination language hypotheses for source language input material based on a translation model parameter, and operation (f) comprises modifying the translation model parameter.
17. The apparatus of claim 13 , wherein the one or more models comprise a language model that selects between two or more destination language hypotheses for source language input material based on a language model parameter, and operation (f) comprises modifying the language model parameter.