Methods and systems for facilitating application programming interface communications
Method and systems for facilitating communications using application programming interfaces (“APIs”) by interpreting a received command based on the command and an image of the user interface that was displayed on a display screen when the command was received.
1. A method for facilitating communications using application programming interfaces (“APIs”), the method comprising:
receiving, from a device, via a network, by control circuitry of a server, an application programming interface (“API”) request for interpreting a command, wherein the API request includes an image of a user interface as displayed on a display screen of the device when the command was received, wherein the server is remote from the device;
caching, by the control circuitry, at the server, the image in the API request;
determining, by the control circuitry of the server, a command response based on the command and the image;
generating, by the server, an API response based on the command response; and
transmitting, from the server, via the network, the API response to the device.
2. The method of claim 1 , wherein determining the command response based on the command and the image further comprises:
determining an object in the image;
determining a context for the user interface based on the object; and
customizing the command response based on the context.
3. The method of claim 2 , wherein the object in the image is determined by:
determining boundaries of objects in the image;
matching the boundaries of objects to a user interface template of a plurality of user interface templates, wherein each of the plurality of user interface templates corresponds to a respective context; and
determining the context for the user interface based on the respective context for the user interface template.
4. The method of claim 1 , wherein determining the command response based on the command and the image further comprises:
determining an object in the image;
determining a position of the object in the user interface; and
customizing the command response based on the position.
5. The method of claim 1 , wherein determining the command response based on the command and the image further comprises:
determining an object in the image;
determining a word corresponding to the object in the user interface; and
customizing the command response based on the word.
6. The method of claim 1 , wherein determining the command response based on the command and the image comprises interpreting the command based on an object in the image.
7. The method of claim 1 , wherein the command is a vocal search command, and the API request is for a voice recognition application.
8. The method of claim 1 , further comprising transmitting, by the control circuitry, the API request from a first device to a second device.
9. The method of claim 1 wherein the API response is customized based on the image.
10. The method of claim 1 , wherein the image is captured prior to modifying the user interface in response to the command.
11. A system for facilitating communications using application programming interfaces (“APIs”), the system comprising:
control circuitry configured to:
receive, from a device, via a network, at a server, an application programming interface (“API”) request for interpreting a command, wherein the API request includes an image of a user interface as displayed on a display screen of the device when the command was received, wherein the server is remote from the device;
cache, at the server, the image in the API request;
determine, at the server, a command response based on the command and the image; and
generate, at the server, an API response based on the command response; and
output circuitry configured to transmit the API response from the server via the network to the device.
12. The system of claim 11 , wherein the control circuitry is further configured to:
determine an object in the image;
determine a context for the user interface based on the object; and
customize the command response based on the context.
13. The system of claim 12 , wherein the control circuitry is further configured to:
determine boundaries of objects in the image;
match the boundaries of objects to a user interface template of a plurality of user interface templates, wherein each of the plurality of user interface templates corresponds to a respective context; and
determine the context for the user interface based on the respective context for the user interface template.
14. The system of claim 11 , wherein the control circuitry is further configured to:
determine an object in the image;
determine a position of the object in the user interface; and
customize the command response based on the position.
15. The system of claim 11 , wherein the control circuitry is further configured to:
determine an object in the image;
determine a word corresponding to the object in the user interface; and
customize the command response based on the word.
16. The system of claim 11 , wherein determining the command response based on the command and the image comprises interpreting the command based on an object in the image.
17. The system of claim 11 , wherein the command is a vocal search command, and the API request is for a voice recognition application.
18. The system of claim 11 , wherein the API request is received, by a second device, from a first device.
19. The system of claim 11 wherein the API response is customized based on the image.
20. The system of claim 11 , wherein the image is captured prior to modifying the user interface in response to the command.