IP Library Granted Patent US 12,141,542
Granted Patent B2
US 12,141,542 · App. 17/553,798 · Granted Nov 12, 2024

Methods and systems for creating a training dataset for training a Machine Learning Algorithm (MLA) for a machine-translation task

Inventors: Anton Aleksandrovich Dvorkovich (Moscow, RU); Roman Olegovich Peshkurov (g Kolomna, RU)
Assignee: Y.E. Hub Armenia LLC
G06F40/58G06F40/44
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,141,542
App. No.
17/553,798
Granted
Nov 12, 2024
Kind
B2
Abstract

Methods and servers for training a translation model for translation between a rare language from a group and a target language. The method includes acquiring an actual example of translation and using a transliteration function for generating a synthetic actual example of translation. The method includes acquiring a sentence in the target language, generating an artificial translation of that sentence using back-translation, and thereby generating a given artificial example of translation. The method includes generating a synthetic artificial example based on the given artificial example. The method includes training the translation model based on the synthetic actual example of translation and the synthetic artificial example of translation.

Claims (78)

1. A method of training a machine learning translation model for performing translation between a rare language from a group of languages and a target language, the method executable by at least one server, the at least one server having access to a plurality of actual examples of parallel sentences between the group of languages and the target language, the method comprising:

acquiring, by the at least one server, a given actual example having a first sentence in one of the group of languages and a respective second sentence in the target language being an actual translation of the first sentence;

generating, by the at least one server employing a transliteration function, a first synthetic sentence based on the first sentence for the given actual example, the first synthetic sentence being a transliteration of the first sentence into a synthetic writing system,

thereby generating a synthetic actual example having the first synthetic sentence and the respective second sentence;

acquiring, by the at least one server, a third sentence in the target language;

generating, by the at least one server employing another machine learning translation model, a fourth sentence in the rare language for the third sentence using back-translation, the fourth sentence being an artificial translation of the third sentence, the fourth sentence and the third sentence forming a given artificial example;

generating, by the at least one server employing the transliteration function, a fourth synthetic sentence based the fourth sentence for the given artificial example, the fourth synthetic sentence being a transliteration of the fourth sentence into the synthetic writing system,

thereby generating a synthetic artificial example having the fourth synthetic sentence and the third sentence;

generating, by the at least one server, a training dataset comprising the synthetic artificial example and the synthetic actual example, wherein the training dataset comprises a first label indicating that the synthetic actual example is based on an actual translation, and a second label indicating that the synthetic artificial example is based on an artificial translation;

training, by the at least one server, the machine learning translation model based on the training dataset, wherein the training comprises discriminating between artificial and actual translations using the first and the second label, wherein the first label and the second label minimize an effect of training iterations performed so that the synthetic artificial example has a lower effect on the machine learning translation model than the synthetic actual example;

receiving a given in-use sentence in the rare language;

generating a synthetic in-use sentence based on the given in-use sentence by employing the transliteration function; and

translating, using the trained machine learning translation model, the in-use synthetic sentence into a respective in-use sentence in the target language.

2. The method of claim 1 ,

wherein the generating the synthetic actual example comprises:

generating, by the at least one server, a plurality of synthetic actual examples for respective ones from the plurality of actual examples;

and wherein the third sentence is one of a plurality of third sentences and wherein the generating the synthetic artificial example comprises:

generating, by the at least one server, a plurality of synthetic artificial examples for respective ones from a plurality of artificial examples;

and wherein the training the machine learning translation model comprises:

training, by the at least one server, the machine learning translation model based on the plurality of synthetic actual examples and the plurality of synthetic artificial examples.

3. The method of claim 1 , wherein the method further comprises:

generating, by the at least one server, the transliteration function by training a Statistical Machine Translation (SMT) model based on aligned corpuses of text between languages in the group of languages,

the SMT model being trained for determining transliteration rules between sentences in each one of the group of languages and synthetic sentences in the synthetic writing system.

4. The method of claim 1 , wherein the group of languages is a Turkic group of languages, the Turkic group of languages comprising Turkish language, Uzbek language, Chuvash language, and Kyrgyz language, the rare language being the Chuvash language.

5. The method of claim 1 , wherein the group of languages is a Germanic group of languages, the Germanic group of languages comprising English language, German language, Dutch language, and Faroese language, the rare language being the Faroese language.

6. The method of claim 1 , wherein the group of languages is a Romance group of languages, the Romance group of languages comprising French language, Italian language, Spanish language, and Galician language, the rare language being the Galician language.

7. The method of claim 1 ,

wherein the other machine learning translation model is trained based on the actual examples between the rare language and the target language.

8. The method of claim 1 ,

wherein the method further comprises:

receiving, by the at least one server, the given in-use sentence in the rare language;

generating, by the at least one server employing the transliteration function, the synthetic in-use sentence based on the given in-use sentence; and

generating, by the at least one server employing the machine learning translation model, the respective in-use sentence in the target language based on the in-use synthetic sentence.

9. The method of claim 1 ,

wherein the other machine learning translation model is the machine learning translation model, and wherein the training comprises:

training, by the at least one server, the machine learning translation model based on the synthetic actual example;

generating, by the at least one server employing the machine learning translation model, the fourth sentence in the rare language for the third sentence using back-translation; and

further training, by the at least one server, the machine learning translation model based on the synthetic artificial example.

10. The method of claim 1 , wherein the third sentence is a clean sentence in the target language, the clean sentence being pre-selected by a human operator.

11. A system for training a machine learning translation model for performing translation between a rare language from a group of languages and a target language, the system having access to a plurality of actual examples of parallel sentences between the group of languages and the target language, the system comprising at least one processor and memory comprising executable instructions which, when executed by the at least one processor, cause the system to:

acquire a given actual example having a first sentence in one of the group of languages and a respective second sentence in the target language being an actual translation of the first sentence;

generate, by employing a transliteration function, a first synthetic sentence based on the first sentence for the given actual example, the first synthetic sentence being a transliteration of the first sentence into a synthetic writing system,

thereby generating a synthetic actual example having the first synthetic sentence and the respective second sentence;

acquire a third sentence in the target language;

generate, by employing another machine learning translation model, a fourth sentence in the rare language for the third sentence using back-translation, the fourth sentence being an artificial translation of the third sentence, the fourth sentence and the third sentence forming a given artificial example;

generate, by employing the transliteration function, a fourth synthetic sentence based the fourth sentence for the given artificial example, the fourth synthetic sentence being a transliteration of the fourth sentence into the synthetic writing system,

thereby generating a synthetic artificial example having the fourth synthetic sentence and the third sentence;

generate a training dataset comprising the synthetic artificial example and the synthetic actual example, wherein the training dataset comprises a first label indicating that the synthetic actual example is based on an actual translation, and a second label indicating that the synthetic artificial example is based on an artificial translation; and

train the machine learning translation model based on the training dataset, wherein the training comprises discriminating between artificial and actual translations using the first and the second label, wherein the first label and the second label minimize an effect of training iterations performed so that the synthetic artificial example has a lower effect on the machine learning translation model than the synthetic actual example;

receive a given in-use sentence in the rare language;

generate a synthetic in-use sentence based on the given in-use sentence by employing the transliteration function; and

translate using the machine learning translation model the in-use synthetic sentence into a respective in-use sentence in the target language.

12. The system of claim 11 ,

wherein the instructions that cause the system to generate the synthetic actual example comprises instructions that cause the system to:

generate a plurality of synthetic actual examples for respective ones from the plurality of actual examples;

wherein the third sentence is one of a plurality of third sentences and wherein the instructions that cause the system to generate the synthetic artificial example comprise instructions that cause the system to:

generate a plurality of synthetic artificial examples for respective ones from a plurality of artificial examples;

and wherein the instructions that cause the system to train the machine learning translation model comprise instructions that cause the system to:

train the machine learning translation model based on the plurality of synthetic actual examples and the plurality of synthetic artificial examples.

13. The system of claim 11 , wherein the instructions further cause the system to:

generate the transliteration function by training a Statistical Machine Translation (SMT) model based on aligned corpuses of text between languages in the group of languages,

the SMT model being trained for determining transliteration rules between sentences in each one of the group of languages and synthetic sentences in the synthetic writing system.

14. The system of claim 11 , wherein the group of languages is a Turkic group of languages, the Turkic group of languages comprising Turkish language, Uzbek language, Chuvash language, and Kyrgyz language, the rare language being the Chuvash language.

15. The system of claim 11 , wherein the group of languages is one of:

a Germanic group of languages, the Germanic group of languages comprising English language, German language, Dutch language, and Faroese language, the rare language being the Faroese language,

a Romance group of languages, the Romance group of languages comprising French language, Italian language, Spanish language, and Galician language, the rare language being the Galician language.

16. The system of claim 11 ,

wherein the other machine learning translation model is trained based on the actual examples between the rare language and the target language.

17. The system of claim 11 ,

wherein the instructions further cause the system to:

receive the given in-use sentence in the rare language;

generate, by employing the transliteration function, the synthetic in-use sentence based on the given in-use sentence; and

generate, by employing the machine learning translation model, the respective in-use sentence in the target language based on the in-use synthetic sentence.

18. The system of claim 11 ,

wherein the other machine learning translation model is the machine learning translation model, and wherein the instructions further cause the system to:

train the machine learning translation model based on the synthetic actual example;

generate, by employing the machine learning translation model, the fourth sentence in the rare language for the third sentence using back-translation; and

further train the machine learning translation model based on the synthetic artificial example.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2024
From: DIRECT CURSUS TECHNOLOGY L.L.C
To: Y.E. HUB ARMENIA LLC
Reel/Frame 068534/0687 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2023
From: YANDEX EUROPE AG
To: DIRECT CURSUS TECHNOLOGY L.L.C
Reel/Frame 065692/0720 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2023
From: DVORKOVICH, ANTON ALEKSANDROVICH; PESHKUROV, ROMAN OLEGOVICH
To: YANDEX.TECHNOLOGIES LLC
Reel/Frame 062415/0521 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2023
From: YANDEX.TECHNOLOGIES LLC
To: YANDEX LLC
Reel/Frame 062415/0562 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2023
From: YANDEX LLC
To: YANDEX EUROPE AG
Reel/Frame 062415/0581 →
Priority Claims (1)
RU RU2020142417 · Dec 22, 2020 · national
Continuity (1)
Related Publication 20220198159A1 · Jun 23, 2022