Suggesting actions based on machine learning
This document describes techniques for suggesting actions based on machine learning. These techniques determine a task that a user desires to perform, and presents a user interface through which to perform the task. To determine this task, the techniques can analyze content displayed on the user device or analyze contexts of the user and user device. With this determined task, the techniques determine an action that may assist the user in performing the task. This action is further determined to be performable through analysis of functionalities of an application, which may or may not be executing or installed on the user device. With some subset of the application's functionalities determined, the techniques presents the subset of functionalities via the user interface. By so doing, the techniques enable a user to complete a task more easily, quickly, or using fewer computing resources.
1. A computer-implemented method, comprising:
displaying, at a present time of a present day, content in a window occupying a portion of a display of a user device, the window associated with a first application;
predicting, based on an association between the displayed content in the window associated with the first application and a context related to a past user interaction, a suggested action to be performed by the user via a second application, the past user interaction having occurred on a prior day and at a prior time that was temporally proximate to the present time of the present day; and
displaying a limited version of the second application through a slice window, the limited version comprising a subset of functionalities of the second application capable of performing the suggested action, and the slice window occupying another portion of the display and including one or more selectable controls through which to instruct the second application to perform the suggested action.
2. The method of claim 1 , wherein the context is associated with the user, the user device, or both.
3. The method of claim 2 , wherein the context is based on one or more of time, location, situation, or activity.
4. The method of claim 1 , wherein the context is detected by sensors or circuitry that are local to the user device.
5. The method of claim 1 , wherein the context is received from a device different from the user device.
6. The method of claim 1 , further comprising:
determining an association between the context and a prior context associated with the user, the user device, or both, and
wherein the suggested action is based on the determined association between the context and the prior context.
7. The method of claim 1 , further comprising:
determining an association between the context and another context associated with another user, another user device, or both, and
wherein the suggested action is based on the determined association between the context and the other context.
8. The method of claim 1 , wherein the determining of the context is performed by a machine learning model, and wherein the predicting of the suggested action comprises predicting, by a machine learning model and based on the context, the suggested action.
9. The method of claim 8 , wherein the determining of the context by the machine learning model is further based on text content analyzed via text recognition, image content analyzed via image processing, or audio content analyzed via audio processing, the text content, image content, or audio content associated with a window occupying a portion of the display of the user device.
10. The method of claim 1 , wherein the slice window is a limited-function user interface for the second application.
11. The method of claim 1 , wherein the content comprises at least one of text content, image content, content from a book, music content, or video content.
12. The method of claim 1 , further comprising:
associating the user to the user device, wherein the associating of the user to the user device is based on a facial recognition of the user, a voice recognition of the user, a radio frequency identifier (RFID) tag associated with a user, a retinal scan, or a login to the user device by the user.
13. The method of claim 1 , further comprising:
receiving a selection of the one or more selectable controls by the user; and
responsive to the receiving of the selection of the one or more selectable controls, displaying second content, the second content including a result of the second application performing the suggested action.
14. The method of claim 13 , wherein the displaying of the second content comprises displaying the second content in another window occupying at least a portion of the display of the user device.
15. The method of claim 14 , wherein the other window comprises another slice window.
16. The method of claim 14 , wherein the other window is associated with the second application.
17. The method of claim 1 , wherein the suggested action is an installation of the second application, and the method further comprising:
performing the installation of the second application prior to the displaying of the limited functionalities of the second application through the slice window.
18. The method of claim 1 , wherein the suggested action comprises at least one of translating text, translating audio content, converting currency, reading text-to-speech, auto-filling passwords, capturing an image, creating an event, modifying an event, creating a contact list entry, modifying a contact list entry, contacting an individual, purchasing media, purchasing a ticket to a media event or providing codes.
19. A user device comprising:
a display;
a processor; and
non-transitory computer-readable storage media storing instructions that, upon execution by the processor, cause the user device to:
display, at a present time of a present day, content in a window occupying a portion of the display of the user device, the window associated with a first application;
predict, based on an association between the displayed content in the window associated with the first application and a context related to a past user interaction, a suggested action to be performed by the user via a second application, the past user interaction having occurred on a prior day and at a prior time that was temporally proximate to the present time of the present day; and
display a limited version of the second application through a slice window, the limited version comprising a subset of functionalities of the second application capable of performing the suggested action, and the slice window occupying another portion of the display and including one or more selectable controls through which to instruct the second application to perform the suggested action.
20. A system comprising:
a processor; and
non-transitory computer-readable storage media storing instructions that, upon execution by the processor, cause a user device to:
display, at a present time of a present day, content in a window occupying a portion of a display of the user device, the window associated with a first application;
predict, based on an association between the displayed content in the window associated with the first application and a context related to a past user interaction, a suggested action to be performed by the user via a second application, the past user interaction having occurred on a prior day and at a prior time that was temporally proximate to the present time of the present day; and
display a limited version of the second application through a slice window, the limited version comprising a subset of functionalities of the second application capable of performing the suggested action, and the slice window occupying another portion of the display and including one or more selectable controls through which to instruct the second application to perform the suggested action.