IP Library Granted Patent US 11,769,019
Granted Patent B1
US 11,769,019 · App. 16/953,205 · Granted Sep 26, 2023

Machine translation with adapted neural networks

Inventors: Prashant Mathur (New York, NY); Georgiana Dinu (New York, NY); Anna Currey (New York, NY); Eric J. Nowell (Kirkland, WA); Aakash Upadhyay (Seattle, WA); Haiyu Yao (Seattle, WA); Marcello Federico (Trento, IT); Yaser Al-Onaizan (Cortlandt Manor, NY); Rama Krishna Sandeep Pokkunuri (Redmond, WA); Jian Wang (Bellevue, WA); Xianglong Huang (Bellevue, WA)
Assignee: Amazon Technologies, Inc.
G06F40/58G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,019
App. No.
16/953,205
Granted
Sep 26, 2023
Kind
B1
Abstract

A translation system receives examples of translations between a first language and a second language. In response to receiving request to translate a source text from the first language to the second language, the system ranks the examples based on the example's applicability to one or more portions of the source text. The system performs additional training of a neural network that was pre-trained to translate from the first language to the second language, where the additional training is based on one or more top-ranking examples. The system translates the source text to the second language using the additionally trained neural network.

Claims (51)

1. A system, comprising:

at least one processor; and

at least one memory, the at least one memory comprising instructions that, in response to execution by the at least one processor, cause the system to at least:

receive two or more examples of translation, wherein an example of translation comprises a first phrase in a first language and a second phrase in a second language, the second phrase a translation of the first phrase;

receive a request to translate a source text from the first language to the second language;

rank the two or more examples with respect to a plurality of examples based, at least in part, on a measure of similarity between the first phrase and one or more portions of the source text;

perform additional training of a neural network trained to translate from the first language to the second language, the additional training based at least in part on one or more top-ranking examples of the two or more examples, and at least a portion of an encoder of the neural network is unchanged during the additional training of the neural network; and

translate, based at least in part on the additionally trained neural network, the source text from the first language to the second language.

2. The system of claim 1 , wherein the system, in response to receiving the request to translate the source text, causes allocation of computing capacity, by one or more computing services, for ranking the two or more examples and performing the additional training.

3. The system of claim 1 , wherein the measure of similarity is calculated based at least in part on identification of a sequence of at least one of characters or words in the first phrase that have a corresponding sequence the source text.

4. The system of claim 1 , the at least one memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:

calculate a learning rate based, at least in part, on the measure of similarity;

calculate a number of epochs based, at least in part, on the measure of similarity; and

perform the additional training based, at least in part, on the learning rate and the number of epochs.

5. The system of claim 1 , the at least one memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:

translate a first portion of the source text using the additionally trained neural network; and

translate a second portion of the source text using a version of the neural network that has not been additionally trained.

6. A method, comprising:

receiving one or more examples of translations between a first language and a second language;

receiving a request to translate a source text from the first language to the second language;

ranking the one or more examples based, at least in part, on applicability of at least the first language of the one or more examples to one or more portions of the source text;

performing additional training of a neural network trained to translate from the first language to the second language, the additional training based at least in part on selecting one or more top-ranking examples of the one or more ranked examples, and at least a portion of an encoder of the neural network is unchanged during the additional training of the neural network; and

translating, based at least in part on the additionally trained neural network, the source text from the first language to the second language.

7. The method of claim 6 , further comprising:

communicating, in response to receiving the request to translate the source text, with one or more services to request allocation of computing capacity for ranking the one or more examples.

8. The method of claim 6 , wherein the one or more examples are loaded from a storage service in response to receiving the request to translate the source text.

9. The method of claim 6 , wherein an example, of the one or more examples, comprises a phrase in the first language and a corresponding translation of the phrase in the second language.

10. The method of claim 6 , further comprising:

generating a score indicative of similarity between a first language portion of one of the one or more examples and a portion of the source text, wherein the ranking of the one or more examples is based, at least in part, on the score.

11. The method of claim 6 , further comprising:

determining, based at least in part on a score indicative of the applicability of the one or more of the one or more examples to the one or more portions of the source text, a learning rate and a number of epochs for the additional training, wherein the additional training is performed based at least in part on the learning rate and the number of epochs.

12. The method of claim 6 , further comprising:

translating a first portion of the source text using the additionally trained neural network; and

translating a second portion of the source text using a second neural network.

13. The method of claim 6 , wherein a first portion of the source text is translated using the additionally trained neural network based, at least in part, on an applicability score of the one or more of the one or more examples being above a threshold amount.

14. The method of claim 6 , wherein a second portion of the source text is translated using a neural network that has not been additionally trained using the one or more examples.

15. A non-transitory computer-readable storage medium comprising instructions that, in response to execution by at least one processor of a computing device, cause the computing device to at least:

receive a request to translate a source text from a first language to a second language;

load, from storage, one or more examples of translations between the first language and the second language;

rank the two or more examples based, at least in part, on applicability of at least the first language of the one or more of the two or more examples to one or more portions of the source text;

perform additional training of a neural network trained to translate from the first language to the second language, the additional training based at least in part on one or more top-ranking examples of the two or more examples, and at least a portion of an encoder of the neural network is unchanged during the additional training of the neural network; and

translate, based at least in part on the additionally trained neural network, the source text from the first language to the second language.

16. The non-transitory computer-readable medium of claim 15 , wherein the applicability is determined based at least in part on n-grams in the two or more examples of translation that correspond to n-grams in the source text.

17. The non-transitory computer-readable medium of claim 15 , comprising further instructions that, in response to execution by the at least one processor, cause the computing device to at least:

calculate a learning rate based, at least in part, on a measure of the applicability of the one or more of the two or more examples to one or more portions of the source text;

calculate a number of epochs based, at least in part, on the measure of applicability; and perform the additional training based, at least in part, on the learning rate and the number of epochs.

18. The non-transitory computer-readable medium of claim 15 , wherein a portion of the neural network is frozen during the additional training.

19. The non-transitory computer-readable medium of claim 15 , wherein a plurality of portions of the source text are translated, in parallel, using a plurality of additionally trained neural networks.

20. The non-transitory computer-readable medium of claim 15 , comprising further instructions that, in response to execution by the at least one processor, cause the computing device to at least:

translate a first portion of the source text using the additionally trained neural network; and

translate a second portion of the source text using a version of the neural network that has not been additionally trained.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2020
From: MATHUR, PRASHANT; DINU, GEORGIANA; CURREY, ANNA; NOWELL, ERIC J.; UPADHYAY, AAKASH; YAO, HAIYU; FEDERICO, MARCELLO; AL-ONAIZAN, YASER; POKKUNURI, RAMA KRISHNA SANDEEP; WANG, JIAN; HUANG, XIANGLONG
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 054424/0608 →