IP Library Granted Patent US 10,909,331
Granted Patent B2
US 10,909,331 · App. 16/024,475 · Granted Feb 2, 2021

Implicit identification of translation payload with neural machine translation

Inventors: Stephan Peitz (Aachen, DE); Udhyakumar Nallasamy (Santa Clara, CA); Matthias Paulik (San Jose, CA); Yun Tang (Fremont, CA)
Assignee: Apple Inc.
G06F40/58G10L15/1815G10L15/1822G10L15/22G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,909,331
App. No.
16/024,475
Granted
Feb 2, 2021
Kind
B2
Abstract

Systems and processes for operating an electronic device to train a machine-learning translation system are described. In one process, a first set of training data is obtained. The first set of training data includes at least one payload in a first language and a translation of the at least one payload in a second language. The process further includes obtaining one or more templates for adapting the at least one payload; adapting the at least one payload using the one or more templates to generate at least one adapted payload formulated as a translation request; generating a second set of training data based on the at least one adapted payload; and training the machine-learning translation system using the second set of training data.

Claims (112)

1. An electronic device, comprising:

one or more processors;

a microphone; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving, via a microphone, a speech input;

in response to receiving the speech input, determining whether the received speech input represents a user request for translation;

in accordance with a determination that the received speech input represents a user request for translation, providing a representation of the received speech input to a machine-learning translation system trained with a set of training data, wherein the set of training data comprises at least one adapted payload formulated as a translation request, wherein the at least one adapted payload formulated as a translation request is adapted by:

obtaining a first set of data including at least one payload in at least one source language and at least one corresponding translation into at least one target language; and

using at least one template configured to facilitate formulating each of the at least one payload in the first set of data as a translation request from a corresponding source language to a corresponding target language, generating the set of training data including the at least one adapted payload;

obtaining, using the trained machine-learning translation system, a response to the user request for translation based on the representation of the received speech input; and

providing an audio output corresponding to the obtained response to the user request for translation.

2. The electronic device of claim 1 , wherein determining whether the received speech input represents a user request for translation comprises:

interpreting the received speech input to derive a user intent;

determining whether the user intent is associated with a translation domain; and

in accordance with a determination that the user intent is associated with a translation domain, determining that the received speech input represents a user request for translation.

3. The electronic device of claim 1 , wherein obtaining, using the trained machine-learning translation system, a response to the user request for translation based on the representation of the received speech input comprises:

determining, without parsing the received speech input to identify a to-be-translated payload, whether the user request for translation includes at least one to-be-translated payload and a single target translation language; and

in accordance with a determination that the user request for translation includes at least one to-be-translated payload and a single target translation language, obtaining a translation of the at least one to-be-translated payload.

4. The electronic device of claim 3 , wherein the one or more programs comprise further instructions for:

in accordance with a determination that the user request for translation includes at least one to-be-translated payload and does not include a single target translation language,

identifying, using the machine-learning translation system, a first inquiry to obtain the single target translation language;

outputting an audio of the first inquiry; and

receiving a first user response to the first inquiry.

5. The electronic device of claim 4 , wherein the one or more programs comprise further instructions for:

obtaining a translation of the at least one to-be-translated payload based on the first user response.

6. The electronic device of claim 3 , wherein the one or more programs comprise further instructions for:

in accordance with a determination that the user request for translation includes a single target translation language and does not include at least one to-be-translated payload,

identifying, using the machine-learning translation system, a second inquiry to obtain the to-be-translated payload;

outputting an audio of the second inquiry; and

receiving a second user response to the second inquiry.

7. The electronic device of claim 6 , wherein the one or more programs comprise further instructions for:

obtaining a translation of the at least one to-be-translated payload based on the second user response.

8. The electronic device of claim 3 , wherein the one or more programs comprise further instructions for:

in accordance with a determination that the user request for translation does not include at least one to-be-translated payload and does not include a single target translation language,

identifying, using the machine-learning translation system, a third inquiry to obtain the at least one to-be-translated payload and the single target translation language;

outputting an audio of the third inquiry; and

receiving a third user response to the third inquiry.

9. The electronic device of claim 8 , wherein the one or more programs comprise further instructions for:

obtaining a translation of the at least one to-be-translated payload based on the third user response.

10. A method of performing translation using a machine-learning translation system, the method comprising:

at an electronic device with one or more processors, memory, and a microphone:

receiving, via the microphone, a speech input;

in response to receiving the speech input, determining whether the received speech input represents a user request for translation;

in accordance with a determination that the received speech input represents a user request for translation, providing a representation of the received speech input to a machine-learning translation system trained with a set of training data, wherein the set of training data comprises at least one adapted payload formulated as a translation request, wherein the at least one adapted payload formulated as a translation request is adapted by:

obtaining a first set of data including at least one payload in at least one source language and at least one corresponding translation into at least one target language; and

using at least one template configured to facilitate formulating each of the at least one payload in the first set of data as a translation request from a corresponding source language to a corresponding target language, generating the set of training data including the at least one adapted payload;

obtaining, using the trained machine-learning translation system, a response to the user request for translation based on representation of the received speech input; and

providing an audio output corresponding to the obtained response to the user request for translation.

11. The method of claim 10 , wherein determining whether the received speech input represents a user request for translation comprises:

interpreting the received speech input to derive a user intent;

determining whether the user intent is associated with a translation domain; and

in accordance with a determination that the user intent is associated with a translation domain, determining that the received speech input represents a user request for translation.

12. The method of claim 10 , wherein obtaining, using the trained machine-learning translation system, a response to the user request for translation based on the representation of the received speech input comprises:

determining, without parsing the received speech input to identify a to-be-translated payload, whether the user request for translation includes at least one to-be-translated payload and a single target translation language; and

in accordance with a determination that the user request for translation includes at least one to-be-translated payload and a single target translation language, obtaining a translation of the at least one to-be-translated payload.

13. The method of claim 12 , further comprising:

in accordance with a determination that the user request for translation includes at least one to-be-translated payload and does not include a single target translation language:

identifying, using the machine-learning translation system, a first inquiry to obtain the single target translation language;

outputting an audio of the first inquiry; and

receiving a first user response to the first inquiry.

14. The method of claim 13 , further comprising:

obtaining a translation of the at least one to-be-translated payload based on the first user response.

15. The method of claim 12 , further comprising:

in accordance with a determination that the user request for translation includes a single target translation language and does not include at least one to-be-translated payload:

identifying, using the machine-learning translation system, a second inquiry to obtain the to-be-translated payload;

outputting an audio of the second inquiry; and

receiving a second user response to the second inquiry.

16. The method of claim 15 , further comprising:

obtaining a translation of the at least one to-be-translated payload based on the second user response.

17. The method of claim 12 , further comprising:

in accordance with a determination that the user request for translation does not include at least one to-be-translated payload and does not include a single target translation language:

identifying, using the machine-learning translation system, a third inquiry to obtain the at least one to-be-translated payload and the single target translation language;

outputting an audio of the third inquiry; and

receiving a third user response to the third inquiry.

18. The method of claim 17 , further comprising:

obtaining a translation of the at least one to-be-translated payload based on the third user response.

19. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:

receiving, via the microphone, a speech input;

in response to receiving the speech input, determining whether the received speech input represents a user request for translation;

in accordance with a determination that the received speech input represents a user request for translation, providing a representation of the received speech input to a machine-learning translation system trained with a set of training data, wherein the set of training data comprises at least one adapted payload formulated as a translation request, wherein the at least one adapted payload formulated as a translation request is adapted by:

obtaining a first set of data including at least one payload in at least one source language and at least one corresponding translation into at least one target language; and

using at least one template configured to facilitate formulating each of the at least one payload in the first set of data as a translation request from a corresponding source language to a corresponding target language, generating the set of training data including the at least one adapted payload;

obtaining, using the trained machine-learning translation system, a response to the user request for translation based on the representation of the received speech input; and

providing an audio output corresponding to the obtained response to the user request for translation.

20. The non-transitory computer-readable storage medium of claim 19 , wherein determining whether the received speech input represents a user request for translation comprises:

interpreting the received speech input to derive a user intent;

determining whether the user intent is associated with a translation domain; and

in accordance with a determination that the user intent is associated with a translation domain, determining that the received speech input represents a user request for translation.

21. The non-transitory computer-readable storage medium of claim 19 , wherein obtaining, using the trained machine-learning translation system, a response to the user request for translation based on the representation of the received speech input comprises:

determining, without parsing the received speech input to identify a to-be-translated payload, whether the user request for translation includes at least one to-be-translated payload and a single target translation language; and

in accordance with a determination that the user request for translation includes at least one to-be-translated payload and a single target translation language, obtaining a translation of the at least one to-be-translated payload.

22. The non-transitory computer-readable storage medium of claim 20 , wherein the one or more programs comprise further instructions for:

in accordance with a determination that the user request for translation includes at least one to-be-translated payload and does not include a single target translation language,

identifying, using the machine-learning translation system, a first inquiry to obtain the single target translation language;

outputting an audio of the first inquiry; and

receiving a first user response to the first inquiry.

23. The non-transitory computer-readable storage medium of claim 22 , wherein the one or more programs comprise further instructions for:

obtaining a translation of the at least one to-be-translated payload based on the first user response.

24. The non-transitory computer-readable storage medium of claim 20 , wherein the one or more programs comprise further instructions for:

in accordance with a determination that the user request for translation includes a single target translation language and does not include at least one to-be-translated payload,

identifying, using the machine-learning translation system, a second inquiry to obtain the to-be-translated payload;

outputting an audio of the second inquiry; and

receiving a second user response to the second inquiry.

25. The non-transitory computer-readable storage medium of claim 24 , wherein the one or more programs comprise further instructions for:

obtaining a translation of the at least one to-be-translated payload based on the second user response.

26. The non-transitory computer-readable storage medium of claim 20 , wherein the one or more programs comprise further instructions for:

in accordance with a determination that the user request for translation does not include at least one to-be-translated payload and does not include a single target translation language,

identifying, using the machine-learning translation system, a third inquiry to obtain the at least one to-be-translated payload and the single target translation language;

outputting an audio of the third inquiry; and

receiving a third user response to the third inquiry.

27. The non-transitory computer-readable storage medium of claim 26 , wherein the one or more programs comprise further instructions for:

obtaining a translation of the at least one to-be-translated payload based on the third user response.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2018
From: PEITZ, STEPHAN; NALLASAMY, UDHYAKUMAR; PAULIK, MATTHIAS; TANG, YUN
To: APPLE INC.
Reel/Frame 046728/0588 →
Continuity (2)
Provisional Application 62650969 · Mar 30, 2018
Related Publication 20190303442A1 · Oct 3, 2019