Providing contextual actions for mobile onscreen content
Systems and methods provide an application programming interface to offer action suggestions to third-party applications using context data associated with the third-party. An example method includes receiving content information and context information from a source mobile application, the content information representing information to be displayed on a mobile device as part of a source mobile application administered by a third party, the context information being information specific to the third party and unavailable to a screen scraper. The method also includes predicting an action based on the content information and the context information, the action representing a deep link for a target mobile application. The method further includes providing the action to the source mobile application with a title and a thumbnail, the source mobile application using the title and thumbnail to display a selectable control that, when selected, causes the mobile device to initiate the action.
1. A method implemented by one or more processors, the method comprising:
receiving, via an application programming interface and from a source application that is administered by a third party, and during a current session in which a user is interacting with the source application via a client device:
context information that reflects user interactions with the source application during the current session, wherein the context information is not visible to the user via the client device during the current session;
predicting, based on analysis of the context information, an intent for a target application, wherein the target application differs from the source application, wherein the predicting, based on analysis of the context information, the intent for the target application comprises:
identifying one or more entities based on the context information;
providing the entities as input to a machine learning model trained to predict one or more suggested intents;
receiving output from the machine learning model based on the providing; and
predicting, based on the output, the intent for the target application;
responsive to receiving, during the current session in which the user is interacting with the source application, user input that is directed to the predicted intent for the target application:
initiating the intent with the target application, initiating the intent with the target application including transferring focus from the source application to the target application, and transferring data from the source application to the target application.
2. The method of claim 1 , wherein initiating the intent with the target application comprises initializing the target application in a particular state using the data transferred from the source application to the target application.
3. The method of claim 2 , further comprising:
receiving, via the application programming interface and from the source application that is administered by the third party, and during the current session:
second context information that reflects historical user interactions with the source application during previous sessions,
wherein the predicting is further based on analysis of the second context information.
4. The method of claim 3 , wherein the predicting, based on analysis of the context information and the second context information, the intent for the target application comprises:
identifying one or more entities based on the context information;
providing the entities as input to a machine learning model trained to predict one or more suggested intents;
receiving output from the machine learning model based on the providing; and
predicting, based on the output, the intent for the target application.
5. The method of claim 4 , wherein the output received from the machine learning model comprises multiple intents, the multiple intents including the intent, and wherein predicting, based on the output, the intent, comprises:
ranking the multiple intents based on the context information; and
selecting the intent, from the multiple intents, based on the intent having a highest ranking.
6. The method of claim 1 , wherein the context information further includes user profile data associated with historical use of the source application by the user.
7. The method of claim 6 , wherein the user profile data associated with the historical use of the source application by the user includes at least one of:
user preference data;
user search history data; and
user settings for the source application.
8. The method of claim 1 , further comprising:
receiving, via the application programming interface and from the source application that is administered by the third party, and during the current session:
second context information that reflects historical user interactions with the source application during previous sessions,
wherein the predicting is further based on analysis of the second context information.
9. A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the system to:
receive, via an application programming interface and from a source application that is administered by a third party, and during a current session in which a user is interacting with the source application via a client device:
context information that reflects user interactions with the source application during the current session, wherein the context information is not visible to the user via the client device during the current session;
receive, via the application programming interface and from the source application that is administered by the third party, and during the current session:
second context information that reflects historical user interactions with the source application during previous sessions:
predict, based on analysis of the context information and based on analysis of the second context information, an intent for a target application, wherein the target application differs from the source application;
responsive to receiving, during the current session in which the user is interacting with the source application, user input that is directed to the predicted intent for the target application:
initiate the intent with the target application, initiating the intent with the target application including transferring focus from the source application to the target application, and transferring data from the source application to the target application.
10. The system of claim 9 , wherein initiating the intent with the target application includes initializing the target application in a particular state using the data transferred from the source application to the target application.