Method and apparatus for improving a bilingual corpus, machine translation method and apparatus
According to one aspect, there is provided an apparatus for improving a bilingual corpus including a plurality of sentence pairs of a first language and a second language and word alignment information of each of the sentence pairs, the apparatus comprises: an extracting unit for extracting a split candidate from word alignment information of a given sentence pair; a calculating unit for calculating split confidence of said split candidate; a comparing unit for comparing said split confidence and a pre-set threshold; and a splitting unit for splitting said given sentence pair at said split candidate in a case that said split confidence is larger than said pre-set threshold.
1. A method performed by a machine translation apparatus which improves a bilingual corpus and which performs statistical machine translation based on the improved bilingual corpus, said bilingual corpus including a plurality of sentence pairs of a first language and a second language and word alignment information of each of the sentence pairs, the method comprising operating the machine translation apparatus to perform functions comprising:
extracting a split candidate for splitting a given sentence pair into a plurality of sub-sentence pairs, from word alignment information of the given sentence pair;
calculating a split confidence of said split candidate based on both of a number of word alignments across said split candidate and a length of said given sentence pair, wherein said split confidence is calculated such that as a ratio of said number of word alignments across said split candidate to said length of said given sentence pair decreases, said split confidence increases;
comparing said split confidence and a pre-set threshold;
splitting said given sentence pair at said split candidate in a case in which said split confidence is larger than said pre-set threshold;
re-aligning words of sub-sentence pairs split by said splitting to generate an improved bilingual corpus;
translating an input sentence from one of the first language and the second language to the other of the first language and the second language, based on translation knowledge trained based on the improved bilingual corpus; and
outputting a result of the translating.
2. The method according to claim 1 , wherein a word of said first language and a word of said second language corresponding to said split candidate are at least one of words and characters that are one-to-one aligned and that have sentence segmentation capability.
3. The method according to claim 1 , wherein the bilingual sentence pair is split at a position of a source language sentence and a position of a target language sentence, respectively.
4. The method according to claim 1 , wherein said split confidence is calculated by using a formula as below,
sc
a
j
=
1
-
cross
a
j
sent_len
,
sent_len
=
(
m
+
1
)
/
2
wherein a j is said split candidate, sc a j is said split confidence, cross a j is said number of word alignments across said split candidate, sent_len is said length of said given sentence pair, m is a length of the sentence of said first language of said given sentence pair, and l is a length of the sentence of said second language of said given sentence pair.