Method, apparatus, and system for facilitating cross-application searching and retrieval of content using a contextual user model

1. A content discovery module to retrieve content in an automated fashion at a computing device, comprising executable instructions embodied in one or more non-transitory machine-readable media, configured to cause a computing system comprising one or more computing devices to:

access a contextual user model, the contextual user model comprising:

gaze-tracking data comprising one or more real-time sensor inputs indicative of user gaze in relation to on-screen locations of a display of the computing device; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the gaze-tracking data, the semantic descriptions comprising information about the user interface elements;

generate an inference relating to a user-specific current interaction context based on the contextual user model;

formulate a search query including one or more aspects of the user-specific current interaction context;

execute the search query across one or more software applications; and

present one or more results of the search query at the computing device.

2. The content discovery module of claim 1 , configured to determine, based on the gaze-tracking data, a plurality of recently-viewed user interface elements associated with a plurality of different software applications, determine search criteria based on one or more of the recently-viewed user interface elements, and formulate the search query using the search criteria.

3. The content discovery module of claim 2 , wherein the recently-viewed user interface elements comprise an email message, a calendar appointment, an electronic document, or a web page.

4. The content discovery module of claim 1 , configured to execute the search query across a plurality of networked computing devices.

5. The content discovery module of claim 1 , configured to generate the inference based on a semantic description of a user interface element displayed at an on-screen location corresponding to the user gaze.

6. The content discovery module of claim 5 , configured to incorporate at least a portion of the semantic description into the search query.

7. The content discovery module of claim 6 , wherein the portion of the semantic description incorporated into the search query comprises a keyword or topic derived from the user interface element.

8. The content discovery module of claim 1 , configured to generate the inference based on gaze-tracking data indicating a duration of user gaze at an on-screen location corresponding to a user interface element.

9. The content discovery module of claim 8 configured to incorporate into the search query at least a portion of a semantic description associated with the user interface element located at the on-screen location of user gaze.

10. The content discovery module of claim 1 , configured to generate the inference based on gaze-tracking data indicating an intensity of user gaze at an on-screen location corresponding to a user interface element.

11. The content discovery module of claim 10 , configured to incorporate into the search query at least a portion of a semantic description associated with the user interface element located at the on-screen location of user gaze.

12. The content discovery module of claim 1 , configured to dynamically change the presentation of the results based on the user-specific current interaction context.

13. The content discovery module of claim 1 , configured to present the results at an on-screen location corresponding to gaze-tracking data indicating a duration of user gaze.

14. The content discovery module of claim 1 , configured to present the results at an on-screen location corresponding to gaze-tracking data indicating an intensity of user gaze.

15. The content discovery module of claim 1 , configured to receive user feedback in response to a presented result and modify the presentation of results in response to the user feedback.

16. The content discovery module of claim 1 , wherein the contextual user model comprises application event data from a plurality of software applications and semantic descriptions of the application event data, and the content discovery module is configured to incorporate into the search query at least a portion of a semantic description associated with the application event data.

17. A method for retrieving content in an automated fashion at a computing device, comprising:

maintaining a contextual user model, the contextual user model comprising:

gaze-tracking data comprising one or more real-time sensor inputs indicative of user gaze in relation to on-screen locations of a display of the computing device; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the gaze-tracking data, the semantic descriptions comprising information about the user interface elements;

generating an inference relating to a user-specific current interaction context based on the contextual user model;

formulating a search query including one or more aspects of the user-specific current interaction context;

executing the search query across one or more software applications; and

presenting one or more results of the search query at the computing device.

18. A computing system comprising:

a display;

a sensor subsystem to obtain gaze-tracking data, the gaze-tracking data being indicative of a user's gaze in relation to the display;

one or more processors; and

one or more machine-readable media having stored therein a plurality of instructions that when executed by the processor cause the computing system to perform the method of claim 17 .

19. A cognitive contextual search module to retrieve content in response to user input at a computing device, comprising executable instructions embodied in one or more non-transitory machine-readable media, configured to cause a computing system comprising one or more computing devices to:

receive a search request comprising a data characteristic and a user-specific interaction characteristic;

access a contextual user model, the contextual user model comprising:

gaze-tracking data comprising one or more real-time sensor inputs indicative of a person's gaze in relation to on-screen locations of a display of the computing device; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the gaze-tracking data, the semantic descriptions comprising information about the user interface elements;

perform a search based on the search request and based on the contextual user model to obtain one or more search results; and

present the search results at the computing device.

20. The search module of claim 19 , wherein the data characteristic relates to a software application and the interaction characteristic relates to a different software application.

21. The search module of claim 19 , configured to interpret the search request using one or more of the semantic descriptions from the contextual user model.

22. The search module of claim 19 , wherein the data characteristic comprises a keyword or meta data and the interaction characteristic comprises a reference to a prior user-specific interaction.

23. The search module of claim 22 , configured to map the reference to a prior user-specific interaction to a semantic description contained in the contextual user model.

24. The search module of claim 19 , configured to searchably index a plurality of content with prior interaction information from the contextual user model.

25. The search module of claim 24 , wherein the searchably indexed content comprises content from a plurality of software applications.

26. The search module of claim 24 , wherein the searchably indexed content comprises content located on a plurality of networked computing devices.

27. The search module of claim 19 , configured to dynamically change the presentation of a search result based on the gaze-tracking data.

28. The search module of claim 19 , configured to execute the search query across a plurality of software applications at the computing device.

29. The search module of claim 19 , configured to receive user feedback in response to a presented search result and modify the presentation of the search result in response to the user feedback.

30. The search module of claim 19 , wherein the contextual user model comprises application event data from a plurality of software applications and semantic descriptions of the application event data, and the search module is configured to interpret the search request based on a semantic description associated with the application event data.

31. A method for retrieving content in response to user input at a computing device, comprising:

receiving a search request comprising a data characteristic and a user-specific interaction characteristic;

maintaining a contextual user model, the contextual user model comprising:

gaze-tracking data comprising one or more real-time sensor inputs indicative of a person's gaze in relation to on-screen locations of a display of the computing device; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the gaze-tracking data, the semantic descriptions comprising information about the user interface elements;

interpreting the search request based on the contextual user model;

developing a search query based on the interpreted search request;

executing the search query across one or more software applications to obtain one or more search results; and

presenting the search results at the computing device.

32. A computing system comprising:

a display;

a sensor subsystem to obtain gaze-tracking data, the gaze-tracking data being indicative of a user's gaze in relation to the display;

one or more processors; and

one or more machine-readable media having stored therein a plurality of instructions that when executed by the processor cause the computing system to perform the method of claim 31 .