Method for gestural interpretation in a system for selecting and arranging visible material in document images
View Patent ↗A graphical input and display system for creating and manipulating image object(s) includes input devices permitting a user to select elements of image object(s) received from various image input sources. A processor, connected to the system, receives requests for various image editing operations and also accesses a user interaction module which enables the user to enter new image material or select and modify existing image material without pre-selecting an input mode. The system data memory structure includes a displayed objects data structure, and internal state data structure, and a selected material data structure.
1. A graphical input and display system having a user interface for creating and editing a source electronic image comprising:
user input means for selecting and manipulating image objects;
display means for displaying image objects and system response to user selection actions;
a processor, connected for receiving requests for image editing operations and for accessing a memory structure;
a user interaction module for inferring the intent of user input gestures as entering new image material of arbitrary graphical content or selecting and modifying existing image material without pre-selecting an input mode; and
data memory means comprising:
displayed objects data structure;
internal state data structure; and
selected material data structure.
2. The graphical input and display system having a user interface for creating and editing an electronic image according to claim 1 , wherein said user input means includes a stylus.
3. The graphical input and display system having a user interface for creating and editing an electronic image according to claim 1 , wherein said user input means includes a pen.
4. The graphical input and display system having a user interface for creating and editing an electronic image according to claim 1 , wherein said user input means includes a mouse.
5. The graphical input and display system having a user interface for creating and editing an electronic image according to claim 1 , wherein said selected material data structure includes not less than one mouse/stylus path state and not less than one choice presented state.
6. The graphical input and display system having a user interface for creating and editing an electronic image according to claim 1 , further comprising image input means.
7. A method for utilizing a user interface on a graphical input and display system having user input means, image input means, a processor, and a user interaction module for entering new image material or selecting and modifying existing image material without preselecting an input mode for creating and editing an electronic image object comprising:
receiving a user input gesture;
accessing a user interaction module for inferring the intent of user input gestures as entering new image material of arbitrary graphical content or selecting and modifying existing image material without pre-selecting an input mode;
determining whether said gesture is a selection operation, a draw operation, or an ambiguous operation;
rendering said draw operation as a permanent mark;
servicing said selection operation; and
presenting the user with a choice for interpreting said ambiguous operation.
8. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said user input gesture includes a single tap made on top of or to the side of the electronic image object.
9. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 8 , wherein said single tap is interpreted as a selection gesture.
10. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said user input gesture is interpreted as a drawn dot when said user input gesture includes a single tap made not on top of or to the side of the electronic image object.
11. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 9 , wherein said single tap is interpreted as a drawn dot.
12. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said user input gesture includes a closed or nearly closed path enclosing at least one electronic image object.
13. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said user input gesture is made by pressing of a mouse button.
14. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said user input gesture is made by touching of a stylus or pen to a display screen of the display device.
15. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said servicing said selection operation comprises:
indicating at least one selected object; and
entering a gesture interpretation state.
16. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said ambiguous operation includes a gesture which completely encloses at least one existing electronic image object.
17. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said ambiguous operation includes a gesture which partially encloses at least one existing electronic image object.
18. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said ambiguous operation includes a gesture which forms a line beneath the electronic image object or beside the electronic image object.
19. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said draw operation includes either or both open stroke gestures or closed path gestures which do not enclose an existing electronic image object.
20. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 7 , wherein said choice for interpreting said ambiguous operation includes a pop-up menu choice box.
21. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 20 , further comprising interpreting a tap inside said pop-up menu choice box as a selection operation.
22. The method for utilizing a user interface on a graphical input and display system for creating and editing an electronic image object according to claim 20 , further comprising interpreting a tap outside said pop-up menu choice box as a draw operation.
23. An article of manufacture comprising a computer usable medium having computer readable program code embodied in said medium which, when said program code is executed by said computer causes said computer to perform method steps for invoking draw or select/command operations without setting a user interface mode prior to creating and manipulating a source electronic image, said method steps comprising:
receiving a user input gesture;
accessing a user interaction module for inferring the intent of user input gestures as entering new image material of arbitrary graphical content or selecting and modifying existing image material without pre-selecting an input mode;
determining whether said gesture is a selection operation, a draw operation, or an ambiguous operation;
if said gesture is a draw operation, rendering said draw operation as a permanent mark;
if said gesture is a selection operation, servicing said selection operation; and
if said gesture is an ambiguous operation, presenting the user with a choice for interpreting said ambiguous operation.
24. A method for storing data in a memory for access by a program being executed on a computer for invoking draw or select/command operations without setting a user interface mode prior to creating and manipulating a source electronic image, said method comprising:
storing a displayed objects data structure in said memory;
storing an internal state data structure for inferring the intent of user input gestures as entering new image material of arbitrary graphical content or selecting and modifying existing image material without pre-selecting an input mode, comprising:
storing a mouse/stylus path; and
storing a choice presented state; and
storing a selected material data structure.