Document Mode Processing For Portable Reading Machine Enabling Document Navigation
Controlling a reading machine while reading a document to a user by receiving an image of a document, accessing a knowledge base that provides data that identifies sections in the document and processing user commands to select a section of the document. The reading machine applies text-to-speech to a text file that corresponds to the selected section of the document, to read the selected section of the document aloud to the user.
1 - 20 . (canceled)
21 . A method of operating a portable device, the method comprising:
receiving by a processor in the portable device data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;
processing by the processor the image to recognize a user-initiated gesture in a scene captured in the image;
determining by the processor based on the recognized user-initiated gesture a function to perform by the portable device; and
performing the determined function by the portable device according to the recognized user-initiated gesture.
22 . The method of claim 21 wherein the user selects a gesturing mode to configure the processor, the method further comprising:
determining by the processor that a pointer for providing the gesture was not detected; and
generating by the processor a notification to the user that the pointer was not detected by the portable device.
23 . The method of claim 22 wherein a directed reading mode is selected by the user.
24 . The method of claim 21 wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device includes a separate camera that captures high-resolution images for text recognition processing.
25 . The method of claim 24 wherein processing comprises:
applying pattern-recognition processing to low resolution images captured by cameras on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.
26 . The method of claim 24 wherein the eyeglasses have scanning sensors that produce signals that are translated into signals that allows the user to control the direction of scanning through motion of the user's head.
27 . The method of claim 21 wherein gesturing involves a gesturing item.
28 . The method of claim 27 wherein the gesturing item is a portion of the user's hand or an item held by the user and the gesturing indicates the command from the user to the portable device.
29 . The method of claim 21 wherein commands include: read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures.
30 . The method of claim 26 wherein the command to the portable device to navigate through a hierarchical document, by processing different sections of the document for speech synthesis, is in an order determine by the command.
31 . The method of claim 21 further comprising:
receiving a series of images from the image input device;
processing by the processor in the portable device, the images to extract a series of low resolution images; and
processing by the processor in the portable device the series of low resolution images to detect the presence of the user-initiated gesture in the series of images.
32 . A portable device comprises:
a processor;
memory in communication with the processor, the processor configured to:
receive data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;
process the image to recognize a user-initiated gesture in a scene captured in the image;
determine based on the recognized user-initiated gesture a function to perform by the portable device; and
perform the determined function by the portable device according to the recognized user-initiated gesture.
33 . The portable device of claim 32 wherein the user selects a gesturing mode to configure the processor for gesture processing, the method further comprising:
determine that a pointer for providing the gesture was not detected; and
provide a notification to the user that the pointer was not detected by the portable device.
34 . The portable device of claim 32 wherein a directed reading mode is selected by the user.
35 . The portable device of claim 32 wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device includes a separate camera that captures high-resolution images for text recognition processing.
36 . The portable device of claim 35 wherein the processor is further configured to:
apply pattern-recognition processing to low resolution images captured by low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.
37 . The portable device of claim 32 wherein the commands include read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures.
38 . The portable device of claim 32 wherein the processor is further configured to:
receive a series of images from the image input device;
process the images to extract a series of low resolution images; and
process the series of low resolution images to detect the presence of the user-initiated gesture in the series of images.
39 . A computer program product tangibly stored on a computer readable hardware storage device for controlling a portable electronic device, the computer program product comprises instructions for causing a processor to:
receive data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;
process the image to recognize a user-initiated gesture in a scene captured in the image;
determine based on the recognized user-initiated gesture a function to perform by the portable device; and
perform the determined function by the portable device according to the recognized user-initiated gesture.
40 . The computer program product of claim 39 further comprises instructions to:
apply pattern-recognition processing to low resolution images captured by low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.