Interfacing with applications via dynamically updating natural language processing
Dynamic interfacing with applications is provided. For example, a system receives a first input audio signal. The system processes, via a natural language processing technique, the first input audio signal to identify an application. The system activates the application for execution on the client computing device. The application declares a function the application is configured to perform. The system modifies the natural language processing technique responsive to the function declared by the application. The system receives a second input audio signal. The system processes, via the modified natural language processing technique, the second input audio signal to detect one or more parameters. The system determines that the one or more parameters are compatible for input into an input field of the application. The system generates an action data structure for the application. The system inputs the action data structure into the application, which executes the action data structure.
1 . A method implemented by one or more processors, the method comprising:
receiving, based on an application being activated for execution on a client computing device, a declaration corresponding to the application being activated for execution on the client computing device;
subsequent to the application being activated for execution on the client computing device, and based on the declaration corresponding to the application being activated for execution on the client computing device, modifying a model responsive to the declaration corresponding to the application being activated for execution on the client computing device;
receiving input audio;
processing the input audio, or text recognized from the input audio, using the model modified responsive to the declaration corresponding to the application being activated for execution on the client computing device, to detect one or more parameters;
determining, based on the declaration, that the one or more parameters are compatible for input into the application; and
in response to determining that the one or more parameters are compatible for input into the application, providing the one or more parameters as the input into the application.
2 . The method according to claim 1 , wherein the declaration corresponding to the application being activated for execution on the client computing device identifies functions the application is configured to perform.
3 . The method according to claim 1 , wherein the declaration corresponding to the application being activated for execution on the client computing device identifies a state of a user interface of the application.
4 . The method according to claim 1 , wherein processing the input audio, or the text recognized from the input audio, to detect the one or more parameters comprises using a dictionary selected based on the declaration corresponding to the application being activated for execution on the client computing device.
5 . The method according to claim 1 , wherein providing the one or more parameters as the input into the application comprises:
generating, based on the one or more parameters, an action data structure for the application; and
inputting the action data structure into the application to cause the application to execute the action data structure and update an element of a user interface of the application.
6 . The method according to claim 5 , further comprising:
generating, based on the action data structure, a request for a content item;
transmitting the request for the content item to a remote data processing system; and
responsive to receipt of the content item from the remote data processing system, presenting the content item.
7 . The method according to claim 6 , wherein presenting the content item comprises the application presenting the content item via the user interface of the application.
8 . The method according to claim 6 , wherein presenting the content item comprises presenting the content item via audio output.
9 . A computer program product comprising one or more non-transitory computer-readable storage media having program instructions collectively stored on the one or more non-transitory computer-readable storage media, the program instructions executable to:
receive, based on an application being activated for execution on a client computing device, a declaration corresponding to the application being activated for execution on the client computing device;
subsequent to the application being activated for execution on the client computing device, and based on the declaration corresponding to the application being activated for execution on the client computing device, modify a model responsive to the declaration corresponding to the application being activated for execution on the client computing device;
receive input audio;
process the input audio, or text recognized from the input audio, using the model modified responsive to the declaration corresponding to the application being activated for execution on the client computing device, to detect one or more parameters;
determine, based on the declaration, that the one or more parameters are compatible for input into the application; and
in response to determining that the one or more parameters are compatible for input into the application, provide the one or more parameters as the input into the application.
10 . The computer program product according to claim 9 , wherein the declaration corresponding to the application being activated for execution on the client computing device identifies functions the application is configured to perform.
11 . The computer program product according to claim 9 , wherein the declaration corresponding to the application being activated for execution on the client computing device identifies a state of a user interface of the application.
12 . The computer program product according to claim 9 , wherein processing the input audio, or the text recognized from the input audio, to detect the one or more parameters comprises using a dictionary selected based on the declaration corresponding to the application being activated for execution on the client computing device.
13 . The computer program product according to claim 9 , wherein providing the one or more parameters as the input into the application comprises:
generating, based on the one or more parameters, an action data structure for the application; and
inputting the action data structure into the application to cause the application to execute the action data structure and update an element of a user interface of the application.
14 . The computer program product according to claim 13 , wherein the program instructions are further executable to:
generate, based on the action data structure, a request for a content item;
transmit the request for the content item to a remote data processing system; and
responsive to receipt of the content item from the remote data processing system, present the content item.
15 . The computer program product according to claim 14 , wherein presenting the content item comprises the application presenting the content item via the user interface of the application.
16 . The computer program product according to claim 14 , wherein presenting the content item comprises presenting the content item via audio output.
17 . A system comprising:
a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to:
receive, based on an application being activated for execution on a client computing device, a declaration corresponding to the application being activated for execution on the client computing device;
subsequent to the application being activated for execution on the client computing device, and based on the declaration corresponding to the application being activated for execution on the client computing device, modify a model responsive to the declaration corresponding to the application being activated for execution on the client computing device;
receive input audio;
process the input audio, or text recognized from the input audio, using the model modified responsive to the declaration corresponding to the application being activated for execution on the client computing device, to detect one or more parameters;
determine, based on the declaration, that the one or more parameters are compatible for input into the application; and
in response to determining that the one or more parameters are compatible for input into the application, provide the one or more parameters as the input into the application.
18 . The system according to claim 17 , wherein the declaration corresponding to the application being activated for execution on the client computing device identifies functions the application is configured to perform.
19 . The system according to claim 17 , wherein the declaration corresponding to the application being activated for execution on the client computing device identifies a state of a user interface of the application.
20 . The system according to claim 17 , wherein processing the input audio, or the text recognized from the input audio, to detect the one or more parameters comprises using a dictionary selected based on the declaration corresponding to the application being activated for execution on the client computing device.