Language agnostic command-understanding digital assistant
Systems, methods, and non-transitory computer-readable storage media are provided for crowdsourcing actions and commands of a digital assistant application, irrespective of the languages spoken by users of the digital assistant application. Techniques described herein enable the on-boarding of actions datasets, which include defined commands and actions that result therefrom, from client devices to a remote server device. More specifically, the described techniques facilitate the proper on-boarding, distribution, and retrieval of action datasets regardless of the command language employed by users of the digital assistant application to invoke a properly-determined corresponding action.
1. A computer-implemented method for language agnostic execution of actions, comprising:
receiving, via a microphone coupled to a digital assistant device, a command in a foreign language;
translating the command from the foreign language into a core language;
retrieving, from an action dataset repository, an action dataset of a plurality of action datasets stored therein and determined most relevant to the translated command,
wherein the retrieved action dataset is associated with a parameter definition and is interpretable by the digital assistant device to reproduce a set of inputs utilizing an application of the digital assistant device, and
wherein the retrieved action dataset includes:
the set of inputs reproduced by the application of the digital assistant device, and
multiple command templates, having defined parameter fields, which have been translated from the foreign language to the core language,
a mapping of pieces of electronic content relevant to the multiple command templates and translated to the core language, and
a unique action signature that corresponds to the multiple command templates or the parameter fields; and
causing the digital assistant device to execute the retrieved action dataset based at least in part on the parameter definition.
2. The method of claim 1 , wherein causing the digital assistant device to execute the retrieved action dataset includes causing the digital assistant device to automatically emulate the set of inputs defined at least in part by the retrieved action dataset.
3. The method of claim 2 , wherein the set of inputs includes at least one touch input on the digital assistant device.
4. The method of claim 2 , wherein the set of inputs is defined further in part by a parameter extracted from the command and determined relevant to the parameter definition.
5. The method of claim 4 , wherein the parameter extracted from the command defines a portion of the set of inputs to automatically pass into a form field displayed on the digital assistant device.
6. The method of claim 1 , further comprising:
transcribing the command in the foreign language into alphanumeric text,
wherein the translated command corresponds to the transcribed command after it is translated from the foreign language into the core language.
7. The method of claim 1 , wherein each action dataset of the plurality of action datasets is assigned a corresponding score based a search of the action dataset repository utilizing the translated command, and one action dataset of the plurality of action datasets having a highest corresponding score is determined most relevant to the translated command.
8. The method of claim 1 , further comprising:
generating a search query based on the translated command; and
searching the action dataset repository to determine the most relevant action dataset of the plurality of action datasets,
wherein each action dataset of the plurality of action datasets is associated with a corresponding command in the core language.
9. The method of claim 8 , wherein the command includes a parameter, and the corresponding command associated with the retrieved action dataset includes the parameter definition.
10. A computerized system, comprising:
one or more processors; and
one or more computer storage media storing computer-usable instructions that, when used by the one or more processors, cause the one or more processors to:
receive a command in a foreign language via a microphone coupled to a digital assistant device;
transcribe the command in the foreign language into alphanumeric text;
translate the transcribed command from the foreign language into a core language;
retrieve, from an action dataset repository, an action dataset that includes one of a plurality of actions stored therein and that is determined most relevant to the translated command,
wherein the retrieved action dataset includes:
a set of inputs performable by the digital assistant device,
multiple command templates, having defined parameter fields, which have been translated from the foreign language to the core language,
a mapping of pieces of electronic content relevant of the multiple command templates and translated to the core language, and
a unique action signature that corresponds to the multiple command templates or the parameter fields; and
automatically perform, on the digital assistant device, the set of inputs defined at least in part by the retrieved action dataset.
11. The system of claim 10 , wherein the instructions further cause the one or more processors to:
extract a parameter from the translated command based on a parameter definition defined in the retrieved action dataset, and
wherein the set of inputs is defined further in part by the extracted parameter.
12. The system of claim 11 , wherein a portion of the set of inputs is defined by the extracted parameter and is automatically performed in a form field displayed on the digital assistant device.
13. The system of claim 10 , wherein the instructions further cause the one or more processors to:
employ a language translation service to translate the transcribed command.
14. The system of claim 10 , wherein the instructions further cause the one or more processors to:
generate a search query based on the translated command; and
search the action dataset repository based on the search query to determine the most relevant action dataset of the plurality of actions, wherein each action of the plurality of actions is associated with a corresponding plurality of commands transcribed in the core language.
15. The system of claim 10 , wherein the set of inputs includes at least one touch input.
16. A non-transitory computer storage medium storing computer-useable instructions that, when used by at least one processor, cause the at least one processor to perform operations comprising:
translating a voice command received in a foreign language to a core language that is different than the foreign language;
selecting one action dataset of a plurality of action datasets stored within an action dataset repository and determined most relevant to the translated voice command,
wherein the selected action dataset includes:
a set of inputs to be emulated by an application,
multiple command templates that have been translated from the foreign language to the core language,
a mapping of pieces of electronic content relevant to the multiple command templates and translated to the core language, and
a unique action signature that corresponds to the multiple command templates or the application;
extracting a parameter from the translated voice command based on a parameter definition defined in the selected action dataset; and
automatically emulating, on an application associated with the selected action dataset, a plurality of inputs that is defined based on the selected action dataset and the extracted parameter.
17. The medium of claim 16 , the operations further comprising:
transcribing the translated voice command to alphanumeric text,
wherein the selected action is determined most relevant to the transcribe voice command and the parameter is extracted from the transcribed voice command.
18. The medium of claim 16 , wherein each action dataset of the plurality of action datasets is associated with a corresponding plurality of commands in the core language, and one of the plurality of action datasets is determined most relevant to the translated voice command based at least in part on its corresponding plurality of commands.
19. The medium of claim 18 , wherein one action dataset of the plurality of action datasets is determined most relevant to the translated voice command based further in part on detected contextual data.
20. The medium of claim 16 , wherein each action dataset of the plurality of action datasets corresponds to one action dataset of a plurality of action datasets that defines a corresponding plurality of inputs.