Method and server for performing domain-specific translation
Method and server for performing domain-specific translation of sentences. The method includes generating, by the server, an augmented sequence of input tokens based on an input sentence in the first language and a hint inserted into the input sentence. The method includes iteratively generating, using the NN, a sequence of output tokens based on the augmented sequence of input tokens. The method includes generating a second sentence in the second language using the sequence of output tokens, the second sentence including the domain-specific translation of the given input word.
1. A method of training, by at least one server comprising at least one memory and at least one processor, a transformer model for performing translation from a first language to a second language, the transformer model having an encoder and a decoder, the at least one server executing the transformer model, the method comprising:
during a training phase:
generating, by the at least one server, an augmented sequence of first tokens based on a first sentence in the first language and a first hint inserted into the first sentence,
the first sentence having a first word, the first word being represented in the augmented sequence of first tokens as a given first token, the first hint being represented in the augmented sequence of first tokens as a first start token and a first end token,
the first start token being inserted at a position preceding the given first token and the first end token being inserted at a position following the given first token for identifying which first token in the augmented sequence of first tokens is the given first token;
generating, by the at least one server, an augmented sequence of second tokens based on a second sentence in the second language and a second hint inserted into the second sentence, the second sentence being a translation of the first sentence,
the second sentence having a second word, the second word being a domain-specific translation of the first word, the second word being represented in the augmented sequence of second tokens as a given second token, the second hint being represented in the augmented sequence of second tokens as a second start token and a second end token,
the second start token being inserted at a position preceding the given second token and the second end token being inserted at a position following the given second token for identifying which second token in the augmented sequence of second tokens is the given second token;
training, by the at least one server, the transformer model by providing the augmented sequence of first tokens to the encoder and the augmented sequence of second tokens to the decoder, so as to train the decoder to generate an output start token and an output end token at correct positions in a sequence of output tokens.
2. The method of claim 1 , wherein the method further comprises, during an in-use phase of the transformer model:
iteratively generating, by the at least one server using the transformer model, an augmented sequence of output tokens based on an augmented sequence of input tokens, and during the iteratively generating:
in response to generating an output start token in a sequence of output tokens during a given iteration, applying a constraining condition on a then-next output token to be generated by the transformer model for the augmented sequence of output tokens such that the then-next output token is a domain-specific translation represented as a token;
in response to generating the output end token during another given iteration, stopping the applying the constraining condition on a then-output next token to be generated by the transformer model; and
generating, by the at least one server, a sentence in the second language using the augmented sequence of output tokens.
3. The method of claim 2 , wherein
applying the constraining condition on the then-next output token comprises identifying a plurality of domain-specific translations of an input word corresponding to the augmented sequence of input tokens and wherein the then-next output token corresponds to one of the plurality of domain-specific translations.
4. The method of claim 3 , wherein the plurality of the domain-specific translations of the input word includes grammatical variants of the input word.
5. A server for training a transformer model for performing translation from a first language to a second language, the transformer model having an encoder and a decoder, the server running the transformer model, the server comprising at least one processor and memory comprising executable instructions, which, when executed by the at least one processor, cause the server to:
during a training phase:
generate an augmented sequence of first tokens based on a first sentence in the first language and a first hint inserted into the first sentence,
the first sentence having a first word, the first word being represented in the augmented sequence of first tokens as a given first token, the first hint being represented in the augmented sequence of first tokens as a first start token and a first end token,
the first start token being inserted at a position preceding the given first token and the first end token being inserted at a position following the given first token for identifying which first token in the augmented sequence of first tokens is the given first token;
generate an augmented sequence of second tokens based on a second sentence in the second language and a second hint inserted into the second sentence, the second sentence being a translation of the first sentence,
the second sentence having a second word, the second word being a domain-specific translation of the first word, the second word being represented in the augmented sequence of second tokens as a given second token, the second hint being represented in the augmented sequence of second tokens as a second start token and a second end token,
the second start token being inserted at a position preceding the given second token and the second end token being inserted at a position following the given second token for identifying which second token in the augmented sequence of second tokens is the given second token;
train the transformer model by providing the augmented sequence of first tokens to the encoder and the augmented sequence of second tokens to the decoder, so as to train the decoder to generate an output start token and an output end token at correct positions in a sequence of output tokens.
6. The server of claim 5 , wherein the executable instructions further cause the server to, during an in-use phase of the transformer model:
iteratively generate, using the transformer model, an augmented sequence of output tokens based on an augmented sequence of input tokens, and during the iteratively generating:
in response to generating an output start token in the sequence of output tokens during a given iteration, apply a constraining condition on a then-next output token to be generated by the transformer model for the augmented sequence of output tokens such that the then-next output token is a domain-specific translation represented as a token;
in response to generating the output end token during another given iteration, stop the applying the constraining condition on a then-output next token to be generated by the transformer model; and
generate a second sentence in the second language using the augmented sequence of output tokens.
7. The server of claim 6 , wherein the instructions that cause the server to apply the constraining condition comprise instructions that cause the server to identify a plurality of domain-specific translations of an input word corresponding to the augmented sequence of input tokens.
8. The server of claim 7 , wherein the plurality of the domain-specific translations of the input word includes grammatical variants of the input word.