System and method for training a machine translation system

1. A computer-executable method for training a machine translation system, comprising:

distributing, by a computer, a first translation training task and a training corpus to a first set of servers, wherein a respective first server is configured to:

receive a portion of the training corpus; and

perform the first translation training task on the received portion of the training corpus;

receiving, from the first set of servers, first translation training results; and

distributing a second translation training task and the first translation training results to a second set of servers, wherein a respective second server is configured to:

receive a subset of the first translation training results that correspond to a same language element;

perform the second translation training task on the received subset of the first translation training results; and

output second translation training results.

2. The method of claim 1 ,

wherein the training corpus includes a bilingual corpus, wherein the first translation training task involves one or more of:

computing a forward word-alignment probability indicating a likelihood that a source language word is aligned to a target language word; and

computing a backward word-alignment probability indicating a likelihood that a target language word is aligned to a source language word; and

wherein the first translation training results include one or more of:

the computed forward word-alignment probability; and

the computed backward word-alignment probability.

3. The method of claim 2 , wherein the second translation training task involves one or more of:

computing a forward conditional translation probability indicating a likelihood that a source language word being translated into a target language word; and

computing a backward conditional translation probability indicating a likelihood that a target language word being translated into a source language word.

4. The method of claim 2 , wherein the language element is a source language word if the to-be-distributed first translation training results are forward word-alignment probabilities, and wherein the language element is a target language word if the to-be-distributed first translation training results are backward word-alignment probabilities.

5. The method of claim 2 , further comprising:

distributing the second translation training results to the first set of servers;

in response to the first set of servers determining that the computed forward and/or backward conditional translation probabilities do not converge, updating the forward and/or backward word-alignment probabilities; and

in response to the first set of servers determining that the computed forward and/or backward conditional translation probabilities converge, outputting a word-alignment table based on the computed forward and/or backward conditional translation probabilities.

6. The method of claim 1 ,

wherein the training corpus includes a bilingual corpus;

wherein the first translation training task involves:

extracting, from a sentence pair belonging to the bilingual training corpus, one or more pairs of phrases; and

outputting occurrence frequencies of the phrases; and

wherein the second translation training task involves one of:

computing a forward phrase-translation probability between a source language phrase and a target language phrase; and

computing a backward phrase-translation probability between a target language phrase and a source language phrase.

7. The method of claim 6 , further comprising:

in response to determining that an occurrence frequency of an extracted phrase is less than a predetermined threshold, removing corresponding phrase pairs.

8. The method of claim 1 , wherein the training corpus includes a monolingual corpus, wherein the first translation training task involves performing n-gram segmentations over the monolingual corpus, wherein the first translation training results include segmented n-grams, wherein the second translation training task involves computing n-gram statistics, and wherein the second translation training results include occurrence frequencies of the segmented n-grams.

9. The method of claim 1 , wherein the training corpus includes n-grams obtained from segmenting a monolingual corpus, wherein the first translation training task involves performing prefix segmentation of the n-grams, wherein the first translation training results include prefixes of the n-grams, and wherein the second translation training task involves computing posterior probabilities and/or back-off probabilities of the n-grams.

10. The method of claim 1 , wherein the training corpus includes n-grams obtained from segmenting a monolingual corpus, wherein the first translation training task involves performing suffix segmentation of the n-grams, wherein the first translation training results include suffixes of the n-grams, and wherein the second translation training task involves smoothing probabilities of the n-grams.

11. A computer system, comprising:

a task distribution server;

a plurality of mapping servers; and

a plurality of reduction servers;

wherein the task distribution server is configured to:

distribute a first translation training task and a training corpus to the mapping servers; and

distribute a second translation training task and first translation training results received from the mapping servers to the reduction servers;

wherein a respective mapping server is configured to:

receive a portion of the training corpus; and

perform the first translation training task on the received portion of the training corpus; and

wherein a respective reduction server is configured to:

receive a subset of the first translation training results that correspond to a same language element;

perform the second translation training task on the received subset of the first translation training results; and

output second translation training results.

12. The computer system of claim 11 , wherein the training corpus includes a bilingual corpus, wherein the first translation training task involves one or more of:

computing a forward word-alignment probability indicating a likelihood that a source language word is aligned to a target language word; and

computing a backward word-alignment probability indicating a likelihood that a target language word is aligned to a source language word; and

wherein the first translation training results include one or more of:

the computed forward word-alignment probability; and

the computed backward word-alignment probability.

13. The computer system of claim 12 , wherein the second translation training task involves one or more of:

computing a forward conditional translation probability indicating a likelihood that a source language word being translated into a target language word; and

computing a backward conditional translation probability indicating a likelihood that a target language word being translated into a source language word.

14. The computer system of claim 12 , wherein the language element is a source language word if the to-be-distributed first translation training results are forward word-alignment probabilities, and wherein the language element is a target language word if the to-be-distributed first translation training results are backward word-alignment probabilities.

15. The computer system of claim 12 , wherein the task distribution server is further configured to distribute the second translation training results to the mapping servers, wherein the mapping server is further configured to:

in response to determining that the computed forward and/or backward conditional translation probabilities do not converge, update the forward and/or backward word-alignment probabilities; and

in response to determining that the computed forward and/or backward conditional translation probabilities converge, output a word-alignment table based on the computed forward and/or backward conditional translation probabilities.

16. The computer system of claim 11 , wherein the training corpus includes a bilingual corpus;

wherein the first translation training task involves:

extracting, from a sentence pair belonging to the bilingual training corpus, one or more pairs of phrases; and

outputting occurrence frequencies of the phrases; and

wherein the second translation training task involves one of:

computing a forward phrase-translation probability between a source language phrase and a target language phrase; and

computing a backward phrase-translation probability between a target language phrase and a source language phrase.

17. The computer system of claim 16 , wherein the reduction server is further configured to, in response to determining that an occurrence frequency of an extracted phrase is less than a predetermined threshold, remove corresponding phrase pairs.

18. The computer system of claim 11 , wherein the training corpus includes a monolingual corpus, wherein the first translation training task involves performing n-gram segmentations over the monolingual corpus, wherein the first translation training results include segmented n-grams, wherein the second translation training task involves computing n-gram statistics, and wherein the second translation training results include occurrence frequencies of the segmented n-grams.

19. The computer system of claim 11 , wherein the training corpus includes n-grams obtained from segmenting a monolingual corpus, wherein the first translation training task involves performing prefix segmentation of the n-grams, wherein the first translation training results include prefixes of the n-grams, and wherein the second translation training task involves computing posterior probabilities and/or back-off probabilities of the n-grams.

20. The computer system of claim 11 , wherein the training corpus includes n-grams obtained from segmenting a monolingual corpus, wherein the first translation training task involves performing suffix segmentation of the n-grams, wherein the first translation training results include suffixes of the n-grams, and wherein the second translation training task involves smoothing probabilities of the n-grams.