Crowdsourcing translations
View Patent ↗In one embodiment, a method includes accessing, by one or more of the computing devices, one or more translations for each text string of a plurality of text strings; determining, by one or more of the computing devices, a priority value for each text string of the plurality of text strings, wherein the priority value for the text string is based on one or more reliability-values of the one or more translations for the text string; selecting, by one or more of the computing devices, a particular text string from the plurality of text strings based on its priority value; and sending, to a client system, instructions configured to present a translation prompt comprising the particular text string.
1. A method of crowdsourcing translations, the method comprising, by one or more computing devices:
accessing, by one or more of the computing devices, one or more translations for each text string of a plurality of text strings;
determining, by one or more of the computing devices, a priority value for each text string of the plurality of text strings, wherein the priority value for the text string is based on one or more reliability-values of the one or more translations for the text string, wherein the one or more reliability-values comprise a best-reliability-value of a first text string, wherein the best-reliability-value is a reliability-value of a best translation for the first text string, the best translation having a reliability-value greater than the reliability-values of all other translations for the first text string;
determining that the best-reliability-value of the first text string is within a range of reliability-values below a threshold reliability-value, wherein the threshold reliability-value is a reliability-value above which the best translation is suitable for publishing;
in response to determining that the best-reliability-value of the first text string is within the range of reliability-values below a threshold reliability-value, increasing the priority value for the first text string;
selecting, by one or more of the computing devices, a particular text string from the plurality of text strings based on its priority value; and
sending, from one or more of the computing devices to a client system, instructions configured to display, on a user interface of the client system, a translation prompt comprising the particular text string.
2. The method of claim 1 , wherein a reliability-value of a first translation of the one or more translations is based on a credibility-score of a first user who submitted an input associated with the first translation.
3. The method of claim 2 , wherein the credibility-score of the user who submitted an input associated with the first translation is based on a number of inputs by the first user that match respective correct translations for one or more respective control strings.
4. The method of claim 3 , wherein a correct translation for a control string is a translation that has a reliability-value greater than a threshold reliability-value.
5. The method of claim 2 , further comprising:
accessing a social graph of an online social network, the social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between the two nodes, the nodes comprising:
a first node that corresponds to the first user who submitted the input associated with the first translation;
a plurality of second nodes that each correspond to a second user or a concept; and
calculating a credibility-score of the first user based on social-graph information associated with the first node.
6. The method of claim 5 , wherein the credibility-score of the first user is further based on social-graph information associated with one or more second nodes of the plurality of second nodes.
7. The method of claim 5 , wherein the credibility-score of the first user is further based on a credibility-score of one or more second users corresponding to one or more second nodes of the plurality of second nodes.
8. The method of claim 1 , wherein, for each text string of the plurality of text strings, the priority value of the text string is further based on an impact-metric of the text string, wherein the impact-metric measures an importance of the text string.
9. The method of claim 8 , wherein, for each text string of the plurality of text strings, determining the priority value of the text string further comprises weighting the impact-metric of the text string by a pre-determined value.
10. The method of claim 8 , wherein, for each text string of the plurality of text strings, the impact-metric of the text string is based on an estimate of a number of distinct users for whom a translation of the text string is to be published, the estimate being based on a sample of a plurality of text strings or their associated translations that have been published to a subset of users.
11. The method of claim 8 , wherein, for each text string of the plurality of text strings, the impact-metric of the text string is based on an estimate of an occurrence-frequency of the text string, the occurrence-frequency being a frequency with which the text string or its associated translation is published to individual users of a subset of users.
12. The method of claim 1 , further comprising increasing the priority value of the first text string in response to determining that the best-reliability-value of the first text string is lower than a minimum reliability-value.
13. The method of claim 1 , further comprising increasing the priority value of the first text string in response to determining that the best-reliability-value of the first text string is higher than a pre-determined reliability-value.
14. The method of claim 1 , further comprising publishing the best translation in response to determining that the best-reliability-value exceeds the threshold reliability-value.
15. The method of claim 1 , wherein the translation prompt comprises a first translation-input field associated with a first translation for the particular text string and one or more second translation-input fields associated with one or more second translations for the particular text string, respectively, and wherein, in response to receiving an up-vote for the first translation:
the reliability-value of the first translation is increased by a first amount; and
one or more reliability-values of one or more of the second translations are decreased by a second amount.
16. The method of claim 15 , wherein the first amount and the second amount are determined based on a credibility-score of a user who submitted the up-vote for the first translation.
17. One or more computer-readable non-transitory storage media for crowdsourcing translations, the media embodying software that is operable when executed to:
access, by one or more computing devices, one or more translations for each text string of a plurality of text strings;
determine, by one or more of the computing devices, a priority value for each text string of the plurality of text strings, wherein the priority value for the text string is based on one or more reliability-values of the one or more translations for the text string, wherein the one or more reliability-values comprise a best-reliability-value of a first text string, wherein the best-reliability-value is a reliability-value of a best translation for the first text string, the best translation having a reliability-value greater than the reliability-values of all other translations for the first text string;
determine that the best-reliability-value of the first text string is within a range of reliability-values below a threshold reliability-value, wherein the threshold reliability-value is a reliability-value above which the best translation is suitable for publishing;
in response to determining that the best-reliability-value of the first text string is within the range of reliability-values below a threshold reliability-value, increase the priority value for the first text string;
select, by one or more of the computing devices, a particular text string from the plurality of text strings based on its priority value; and
send, from one or more of the computing devices to a client system, instructions configured to display, on a user interface of the client system, a translation prompt comprising the particular text string.
18. A system for crowdsourcing translations, the system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
access, by one or more computing devices, one or more translations for each text string of a plurality of text strings;
determine, by one or more of the computing devices, a priority value for each text string of the plurality of text strings, wherein the priority value for the text string is based on one or more reliability-values of the one or more translations for the text string, wherein the one or more reliability-values comprise a best-reliability-value of a first text string, wherein the best-reliability-value is a reliability-value of a best translation for the first text string, the best translation having a reliability-value greater than the reliability-values of all other translations for the first text string;
determine that the best-reliability-value of the first text string is within a range of reliability-values below a threshold reliability-value, wherein the threshold reliability-value is a reliability-value above which the best translation is suitable for publishing;
in response to determining that the best-reliability-value of the first text string is within the range of reliability-values below a threshold reliability-value, increase the priority value for the first text string;
select, by one or more of the computing devices, a particular text string from the plurality of text strings based on its priority value; and
send, from one or more of the computing devices to a client system, instructions configured to display, on a user interface of the client system, a translation prompt comprising the particular text string.
19. The media of claim 17 , wherein a reliability-value of a first translation of the one or more translations is based on a credibility-score of a first user who submitted an input associated with the first translation.
20. The media of claim 19 , wherein the credibility-score of the user who submitted an input associated with the first translation is based on a number of inputs by the first user that match respective correct translations for one or more respective control strings.
21. The media of claim 20 , wherein a correct translation for a control string is a translation that has a reliability-value greater than a threshold reliability-value.
22. The media of claim 19 , wherein the software is further operable when executed to:
access a social graph of an online social network, the social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between the two nodes, the nodes comprising:
a first node that corresponds to the first user who submitted the input associated with the first translation;
a plurality of second nodes that each correspond to a second user or a concept; and
calculate a credibility-score of the first user based on social-graph information associated with the first node.
23. The media of claim 22 , wherein the credibility-score of the first user is further based on social-graph information associated with one or more second nodes of the plurality of second nodes.
24. The media of claim 22 , wherein the credibility-score of the first user is further based on a credibility-score of one or more second users corresponding to one or more second nodes of the plurality of second nodes.
25. The media of claim 17 , wherein, for each text string of the plurality of text strings, the priority value of the text string is further based on an impact-metric of the text string, wherein the impact-metric measures an importance of the text string.
26. The media of claim 25 , wherein, for each text string of the plurality of text strings, determining the priority value of the text string further comprises weighting the impact-metric of the text string by a pre-determined value.
27. The media of claim 25 , wherein, for each text string of the plurality of text strings, the impact-metric of the text string is based on an estimate of a number of distinct users for whom a translation of the text string is to be published, the estimate being based on a sample of a plurality of text strings or their associated translations that have been published to a subset of users.
28. The media of claim 25 , wherein, for each text string of the plurality of text strings, the impact-metric of the text string is based on an estimate of an occurrence-frequency of the text string, the occurrence-frequency being a frequency with which the text string or its associated translation is published to individual users of a subset of users.
29. The media of claim 17 , wherein the software is further operable when executed to increase the priority value of the first text string when the best-reliability-value of the first text string is lower than a minimum reliability-value.
30. The media of claim 17 , wherein the software is further operable when executed to increase the priority value of the first text string when the best-reliability-value of the first text string is higher than a pre-determined reliability-value.
31. The media of claim 17 , wherein the software is further operable when executed to publish the best translation in response to determining that the best-reliability-value exceeds the threshold reliability-value.
32. The media of claim 17 , wherein the translation prompt comprises a first translation-input field associated with a first translation for the particular text string and one or more second translation-input fields associated with one or more second translations for the particular text string, respectively, and wherein, in response to receiving an up-vote for the first translation:
the reliability-value of the first translation is increased by a first amount; and
one or more reliability-values of one or more of the second translations are decreased by a second amount.
33. The media of claim 32 , wherein the first amount and the second amount are determined based on a credibility-score of a user who submitted the up-vote for the first translation.