Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device
A device for capturing rendered text is described. The device incorporates one or more visual sensors that receive visual information as a part of capturing rendered text. The visual sensors are collectively capable of capturing both text that is permanently printed on a page, and text that is displayed transitorily on a dynamic device. The device further incorporates a visual information disposition subsystem for disposing of visual information received by the visual sensors. The device further incorporates a package that bears the visual sensors and the visual information disposition subsystem, and is suitable to be held in a human hand.
1. A method comprising:
capturing, at a handheld user device that includes an integrated visual sensor and an integrated visual display, text of a document displayed transitorily on a dynamic display using the visual sensor of the handheld user device;
using textual content of the captured text to identify the document and a position in the document at which a capture is performed;
determining a set of actions appropriate in connection with the capture from the identified document at the identified position, wherein determining the set of actions comprises accessing an electronic counterpart of the identified document and analyzing markup data included in the electronic counterpart of the identified document that associates the identified position with the set of actions;
presenting, on the visual display of the handheld user device, at least a portion of the determined set of actions to a user, wherein each action of the portion of the determined set of actions is presented as a respective menu item of a menu presented on the visual display of the handheld capture device;
receiving, at the handheld capture device, user input selecting one of the presented menu items; and
in response to the received user input, performing the selected action in connection with the capture.
2. The method of claim 1 , wherein the determined set of actions comprises an action to be performed in accordance with a keyword or key phrase that occurs in the identified document in proximity to the identified position.
3. The method of claim 2 , wherein the action to be performed in accordance with the keyword or key phrase comprises providing information associated with a seller of a product related to the keyword or key phrase.
4. The method of claim 1 , wherein the determined set of actions comprises an action for providing help information for the identified document.
5. Non-transitory computer-readable media having contents capable of causing a computing system including one or more computers and a capture device to perform a method, the method comprising:
capturing, at a handheld user device that includes an integrated visual sensor and an integrated visual display, text of a document displayed transitorily on a dynamic display using the visual sensor of the handheld user device;
using textual content of the captured text to identify the document and a position in the document at which a capture is performed;
determining a set of actions appropriate in connection with the capture from the identified document at the identified position, wherein determining the set of actions comprises accessing an electronic counterpart of the identified document and analyzing markup data included in the electronic counterpart of the identified document that associates the identified position with the set of actions;
presenting, on the visual display of the handheld user device, at least a portion of the determined set of actions to a user, wherein each action of the portion of the determined set of actions is presented as a respective menu item of a menu presented on the visual display of the handheld capture device;
receiving, at the handheld capture device, user input selecting one of the presented menu items; and
in response to the received user input, performing the selected action in connection with the capture.
6. The non-transitory computer-readable media of claim 5 , wherein the determined set of actions comprises an action to be performed in accordance with a keyword or key phrase that occurs in the identified document in proximity to the identified position.
7. The non-transitory computer-readable media of claim 6 , wherein the action to be performed in accordance with the keyword or key phrase comprises providing information associated with a seller of a product related to the keyword or key phrase.
8. The non-transitory computer-readable media of claim 5 , wherein the determined set of actions comprises an action for providing help information for the identified document.
9. A system, comprising:
a handheld capture device that includes an integrated visual sensor and an integrated visual display, the handheld capture device configured to capture text of a document displayed transitorily on a dynamic display using the visual sensor; and
one or more computers having one or more memory devices on which are stored computer executable instructions operable to cause the one or more computers to perform operations comprising:
receiving, from the handheld capture device, text of the document displayed transitorily on the dynamic display and captured using the visual sensor of the handheld capture device;
using textual content of the captured text to identify the document and a position in the document at which a capture is performed;
determining a set of actions appropriate in connection with the capture from the identified document at the identified position, wherein determining the set of actions comprises accessing an electronic counterpart of the identified document and analyzing markup data included in the electronic counterpart of the identified document that associates the identified position with the set of actions;
providing, to the handheld capture device for presentation on the visual display of the handheld capture device, at least a portion of the determined set of actions, wherein each action of the portion of the determined set of actions is presented as a respective menu item of a menu presented on the visual display of the handheld capture device;
receiving, at the handheld capture device, user input selecting one of the presented menu items; and
in response to the received user input, performing the selected action in connection with the capture.
10. The system of claim 9 , wherein the determined set of actions comprises an action to be performed in accordance with a keyword or key phrase that occurs in the identified document in proximity to the identified position.
11. The system of claim 10 , wherein the action to be performed in accordance with the keyword or key phrase comprises providing information associated with a seller of a product related to the keyword or key phrase.
12. The system of claim 9 , wherein the determined set of actions comprises an action for providing help information for the identified document.
13. A method comprising:
receiving, at one or more computers and from a handheld user device that includes an integrated visual sensor and an integrated visual display, text of a document displayed transitorily on a dynamic display and captured using the visual sensor of the handheld user device;
using textual content of the captured text to identify the document and a position in the document at which a capture is performed;
determining a set of actions appropriate in connection with the capture from the identified document at the identified position, wherein determining the set of actions comprises accessing an electronic counterpart of the identified document and analyzing markup data included in the electronic counterpart of the identified document that associates the identified position with the set of actions;
providing, to the handheld user device for presentation on the visual display of the handheld user device, at least a portion of the determined set of actions, wherein each action of the portion of the determined set of actions is presented as a respective menu item of a menu presented on the visual display of the handheld capture device;
receiving, at the handheld capture device, user input selecting one of the presented menu items; and
in response to the received user input, performing the selected action in connection with the capture.
14. The method of claim 1 , wherein the determined set of actions comprises an action for printing the electronic counterpart of the identified document and storing a record that includes a timestamp of when the printing occurred and an identifier of a user that selected the action.
15. The method of claim 1 , wherein the determined set of actions comprises an action for presenting the electronic counterpart of the identified document on the visual display of the handheld user device, navigating to the identified position, and highlighting text within the electronic counterpart of the identified document that corresponds to the captured text.
16. The non-transitory computer-readable media of claim 5 , wherein the determined set of actions comprises an action for printing the electronic counterpart of the identified document and storing a record that includes a timestamp of when the printing occurred and an identifier of a user that selected the action.
17. The non-transitory computer-readable media of claim 5 , wherein the determined set of actions comprises an action for presenting the electronic counterpart of the identified document on the visual display of the handheld user device, navigating to the identified position, and highlighting text within the electronic counterpart of the identified document that corresponds to the captured text.
18. The system of claim 9 , wherein the determined set of actions comprises an action for printing the electronic counterpart of the identified document and storing a record that includes a timestamp of when the printing occurred and an identifier of a user that selected the action.
19. The system of claim 9 , wherein the determined set of actions comprises an action for presenting the electronic counterpart of the identified document on the visual display of the handheld user device, navigating to the identified position, and highlighting text within the electronic counterpart of the identified document that corresponds to the captured text.