Systems, methods, means, and media for recording, searching, and outputting display information

1. A system for recording display information, comprising:

a virtual display that:

intercepts display-changes describing changes to be made to a state of a display;

sends the display-changes to a client for display; and

records the display-changes;

a context recorder that uses accessibility mechanisms to record context information derived independently of screen-images,

wherein the accessibility mechanisms include mechanisms utilized by a screen reader for providing desktop access for visually impaired individuals; and

wherein the context information includes (1) text that is displayed in at least one application window, (2) a duration in which the text is displayed on the at least one application window and (3) at least one of a font type of the text, a font size of the text, and an identifier of the at least one application window, that are extracted through the accessibility mechanisms;

a search tool that:

receives a user query from a user;

determines whether the user query matches any of the context information; and

based on the whether the user query matches any of the context information, determines a correspondence between context information that matched the query and at least one of the display changes; and

a display system that:

generates an output screen-image based at least in part on the at least one of the display-changes determined to have a correspondence with the context information that matched the query.

2. The system of claim 1 , wherein the display-changes are display commands.

3. The system of claim 1 , further comprising:

an execution recorder that records execution of an application generating the display; and

an execution revival system that revives execution from a time of the output screen-image.

4. A method for recording display information, comprising:

intercepting display-changes describing changes to be made to a state of a display;

sending the display-changes to a client for display;

recording the display-changes;

recording context information using accessibility mechanisms,

wherein the context information is derived independently of screen-images;

wherein the accessibility mechanisms include mechanisms utilized by a screen reader for providing desktop access for visually impaired individuals; and

wherein the context information includes (1) text that is displayed in at least one application window, (2) a duration in which the text is displayed on the at least one application window and (3) at least one of a font type of the text, a font size of the text, and an identifier of the at least one application window, that are extracted through the accessibility mechanisms;

receiving a user query from a user;

determining whether the user query matches any of the context information; and

based on the whether the user query matches any of the context information, outputting at least one result describing a state of the display generated based at least in part on at least one of the display-changes.

5. The method of claim 4 , wherein the display-changes are display commands.

6. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for recording display information, the method comprising:

intercepting display-changes describing changes to be made to a state of a display;

sending the display-changes to a client for display;

recording the display-changes;

recording context information using accessibility mechanisms,

wherein the context information is derived independently of screen-images;

wherein the accessibility mechanisms include mechanisms utilized by a screen reader for providing desktop access for visually impaired individuals; and

wherein the context information includes (1) text that is displayed in at least one application window, (2) a duration in which the text is displayed on the at least one application window and (3) at least one of a font type of the text, a font size of the text, and an identifier of the at least one application window, that are extracted through the accessibility mechanisms;

receiving a user query from a user;

determining whether the user query matches any of the context information; and

based on the whether the user query matches any of the context information, outputting at least one result describing a state of the display generated based at least in part on at least one of the display-changes.

7. The computer-readable medium of claim 6 , wherein the display-changes are display commands.

8. A system for recording, searching, and outputting display information, comprising:

a display recorder that receives and stores:

display-image-state information including a first screen-image of a display at a first time and a second screen-image of the display at a second time that is after the first time;

display-command information including:

at least one first command that describes advancing the first screen-image to a third screen-image of the display at a third time that is between the first time and the second time; and

at least one second command that describes advancing the second screen-image to a fourth screen-image of the display at a fourth time that is after the third time;

a text recorder that receives:

text displayed in at least one application window at the third time, wherein the text displayed at the third time is extracted independently of screen-images through accessibility mechanisms and wherein the accessibility mechanisms include mechanisms utilized by a screen reader for providing desktop access for visually impaired individuals; and

text displayed in at least one application window at the fourth time, wherein the text displayed at the fourth time is extracted independently of screen-images through the accessibility mechanisms;

a search tool that:

receives a user query; and

determines whether the user query matches the text displayed at the third time or matches the text displayed at the fourth time; and

a display system that, based on whether the user query matches the text displayed at the third time or matches the text displayed at the fourth time;

alters either the first screen-image by applying the at least one first command to generate a third screen-image or alters the second screen-image by applying the at least one second command to generate a fourth screen-image; and

outputs either the third screen-image or the fourth screen-image.

9. The system of claim 8 ,

wherein the display recorder further:

stores the first screen-image with a time stamp related to the first time;

stores the second screen-image with a time stamp related to the second time;

stores the at least one first display command with a time stamp indicating the third time; and

stores the at least one second display command with a time stamp indicating the fourth time; and

wherein the search tool further, based on whether the user query matches the text displayed at the third time or matches the text displayed at the fourth time, determines to either:

alter the first screen-image by applying the at least one first command by determining the first screen-image is a screen-image recorded most closely in time to issuance of the at least one first command based on the time stamp related to the first time and the time stamp related to the second time; or

alter the second screen-image by applying the at least one second command by determining the second screen-image is a screen-image recorded most closely in time to issuance of the at least one second command based on the time stamp related to the second time and the time stamp related to the fourth time.

10. The method of claim 4 , further comprising:

recording execution of an application generating the display; and

reviving execution from a time of the output screen-image.

11. The computer-readable medium of claim 6 , wherein the method further comprising:

recording execution of an application generating the display; and

reviving execution from a time of the output screen-image.