Document mode processing for portable reading machine enabling document navigation
Controlling a reading machine while reading a document to a user by receiving an image of a document, accessing a knowledge base that provides data that identifies sections in the document and processing user commands to select a section of the document. The reading machine applies text-to-speech to a text file that corresponds to the selected section of the document, to read the selected section of the document aloud to the user.
1. A method of operating a portable device, comprising:
receiving by a processor data corresponding to a user-selection of a gesturing mode; said processor activating a gesturing mode configuration based on said user-selection of a gesturing mode;
receiving by a processor in the portable device data that corresponds to an image captured by a camera mounted on a set of eyeglasses, the camera in communication with the portable device;
processing by the processor the image to recognize a user-initiated gesture in a scene captured in the image;
determining by the processor based on the recognized user-initiated gesture a function to perform by the portable device;
performing the determined function by the portable device according to the recognized user-initiated gesture
processing by the processor the image to determine that a pointer for providing the gesture was not detected; and
generating by the processor a notification to the user that the pointer was not detected by the portable device,
wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device comprises a separate camera that captures high-resolution images for text recognition processing.
2. The method of claim 1 , wherein a directed reading mode is selected by the user.
3. The method of claim 1 , wherein processing comprises:
applying pattern-recognition processing to low resolution images captured by cameras on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.
4. The method of claim 3 , wherein the eyeglasses have scanning sensors that produce signals that are translated into signals that allows the user to control the direction of scanning through motion of the user's head.
5. The method of claim 4 , wherein the command to the portable device to navigate through a hierarchical document, by processing different sections of the document for speech synthesis, is in an order determine by the command.
6. The method of claim 1 , wherein the gesturing involves a gesturing item.
7. The method of claim 6 , wherein the gesturing item is a portion of the user's hand or an item held by the user and the gesturing indicates the command from the user to the portable device.
8. The method of claim 1 , wherein commands include: read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures.
9. The method of claim 1 , further comprising:
receiving a series of high resolution images from the camera;
processing by the processor in the portable device, the high resolution images to extract a series of low resolution images; and
processing by the processor in the portable device the series of low resolution images to detect the presence of the user-initiated gesture in the series of images.
10. A portable device comprising:
a processor;
memory in communication with the processor, the processor configured to:
receive data that corresponds to an image captured by an camera mounted on a set of eyeglasses, the camera being in communication with the portable device;
process the image to recognize a user-initiated gesture in a scene captured in the image;
determine based on the recognized user-initiated gesture a function to perform by the portable device; and
perform the determined function by the portable device according to the recognized user-initiated gesture;
wherein the processor is further configured to
receive a user-selection of a gesturing mode to configure the processor for gesture processing,
determine that a pointer for providing the gesture was not detected, and provide a notification to the user that the pointer was not detected by the portable device,
wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device comprises a separate camera that captures high-resolution images for text recognition processing.
11. The portable device of claim 10 wherein a directed reading mode is selected by the user.
12. The portable device of claim 10 , wherein the processor is further configured to:
apply pattern-recognition processing to low resolution images captured by the low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device as commands to the portable device.
13. The portable device of claim 10 , wherein the commands include read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures.
14. The portable device of claim 10 , wherein the processor is further configured to:
receive a series of high resolution images from the camera;
process the high resolution images to extract a series of low resolution images; and
process the series of low resolution images to detect the presence of the user-initiated gesture in the series of images.