Systems and methods for determining translation accuracy in multi-user multi-lingual communications
Various embodiments described herein facilitate multi-lingual communications. The systems and methods of some embodiments enable multi-lingual communications through different modes of communication including, for example, Internet-based chat, e-mail, text-based mobile phone communications, postings to online forums, postings to online social media services, and the like. Certain embodiments implement communication systems and methods that translate text between two or more languages. Users of the systems and methods may be incentivized to submit corrections for inaccurate or erroneous translations, and may receive a reward for these submissions. Systems and methods for assessing the accuracy of translations are described.
1. A method comprising:
performing by one or more computer processors:
obtaining a translation of a message in a first language to a second language wherein the translation was submitted by a user;
generating a part-of-speech (POS) n-gram representation of the translation comprising a plurality of POS n-grams of two or more different lengths;
determining a respective probability for each POS n-gram as a ratio of a count of occurrences of the POS n-gram in a corpus for the second language to a count of all POS n-grams in the corpus having a same length as the POS n-gram; and
determining an accuracy of the translation based on a combination of the probabilities.
2. The method of claim 1 wherein the corpus comprises chatspeak.
3. The method of claim 1 wherein determining the accuracy of the translation based on a combination of the probabilities comprises:
linearly interpolating the probabilities to generate the accuracy.
4. The method of claim 3 wherein linearly interpolating the probabilities to generate the accuracy comprises:
weighting the probabilities of the POS n-grams according to their lengths wherein POS n-grams having lengths greater than other POS n-grams are weighted more than the other POS n-grams.
5. The method of claim 3 , further comprising:
excluding from the interpolation probabilities that do not exceed a threshold.
6. The method of claim 1 wherein generating the POS n-gram representation of the translation comprises:
generating a first POS n-gram from the translation of a first length;
calculating a probability of the first POS n-gram; and
determining that the probability of the first POS n-gram does not exceed a threshold and, based thereon, breaking the first POS n-gram into two or more second POS n-grams having lengths that are less than the first length.
7. The method of claim 1 , further comprising:
revoking the user's translation privileges if the accuracy falls below a threshold value.
8. A system comprising:
a non-transitory computer readable medium having instructions stored thereon; and
at least one processor configured to execute the instructions to perform operations comprising:
obtaining a translation of a message in a first language to a second language wherein the translation was submitted by a user;
generating a part-of-speech (POS) n-gram representation of the translation comprising a plurality of POS n-grams of two or more different lengths;
determining a respective probability for each POS n-gram as a ratio of a count of occurrences of the POS n-gram in a corpus for the second language to a count a all POS n-grams in the corpus having a same length as the POS n-gram; and
determining an accuracy of the translation based on a combination of the probabilities.
9. The system of claim 8 wherein the corpus comprises chatspeak.
10. The system of claim 8 wherein determining the accuracy of the translation based on a combination of the probabilities comprises:
linearly interpolating the probabilities to generate the accuracy.
11. The system of claim 10 wherein linearly interpolating the probabilities to generate the accuracy comprises:
weighting the probabilities of the POS n-grams according to their lengths wherein POS n-grams having lengths greater than other POS n-grams are weighted more than the other POS n-grams.
12. The system of claim 10 , further comprising:
excluding from the interpolation probabilities that do not exceed a threshold.
13. The system of claim 8 wherein generating the POS n-gram representation of the translation comprises:
generating a first POS n-gram from the translation of a first length;
calculating a probability of the first POS n-gram; and
determining that the probability of the first POS n-gram does not exceed a threshold and, based thereon, breaking the first POS n-gram into two or more second POS n-grams having lengths that are less than the first length.
14. The system of claim 8 , wherein the operations further comprise:
revoking the user's translation privileges if the accuracy falls below a threshold value.
15. A manufacture comprising:
non-transitory computer readable media comprising executable instructions, the executable instructions being executable by one or more processors to perform operations comprising:
obtaining a translation of a message in a first language to a second language wherein the translation was submitted by a user;
generating a part-of-speech (POS) n-gram representation of the translation comprising a plurality of POS n-grams of two or more different lengths;
determining a respective probability for each POS n-gram as a ratio of a count of occurrences of the POS n-grams in a corpus for the second language to a count of all POS n-grams in the corpus having a same length as the POS n-gram and
determining an accuracy of the translation based on a combination of the probabilities.
16. The manufacture of claim 15 wherein the corpus comprises chatspeak.
17. The manufacture of claim 15 wherein determining the accuracy of the translation based on a combination of the probabilities comprises:
linearly interpolating the probabilities to generate the accuracy.
18. The manufacture of claim 17 wherein linearly interpolating the probabilities to generate the accuracy comprises:
weighting the probabilities of the POS n-grams according to their lengths wherein POS n-grams having lengths greater than other POS n-grams are weighted more than the other POS n-grams.
19. The manufacture of claim 17 , further comprising:
excluding from the interpolation probabilities that do not exceed a threshold.
20. The manufacture of claim 15 wherein generating the POS n-gram representation of the translation comprises:
generating a first POS n-gram from the translation of a first length;
calculating a probability of the first POS n-gram; and
determining that the probability of the first POS n-gram does not exceed a threshold and, based thereon, breaking the first POS n-gram into two or more second POS n-grams having lengths that are less than the first length.
21. The manufacture of claim 15 , wherein the operations further comprise:
revoking the user's translation privileges if the accuracy falls below a threshold value.