System and method of handwriting recognition in diagrams
A system, method and computer program product for hand-drawing diagrams including text and non-text elements on a computing device are provided. The computing device has a processor and a non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor. Display of input diagram elements in interactive digital ink is performed on a display device associated with the computing device. One or more of the diagram elements are associated with one or more other of the diagram elements in accordance with a class and type of each diagram element. The diagram elements are re-displayed based on one or more interactions with the digital ink received and in accordance with the one or more associations.
1. A system for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the at least one non-transitory computer readable medium configured to:
cause display of a plurality of input diagram elements in interactive digital ink on a display device associated with the computing device, wherein the diagram elements comprise one or more of a geometric shape, a container, a connector, or text;
incrementally associate one or more of the diagram elements with one or more other of the diagram elements in accordance with a class and type of each diagram element;
identify the class of each diagram element by classifying one or more strokes of the hand-drawn input as selected from a group consisting of text, non-text, and junk classifications for each of the one or more strokes; and
incrementally cause re-display of the diagram elements, wherein a display characteristic of a diagram element is influenced based on one or more received interactions with the digital ink, the one or more incremental associations, and a position of the one or more interactions relative to the incrementally associated diagram elements.
2. A system as claimed in claim 1 , wherein the strokes are grouped based on spatial and temporal information of the input.
3. A system as claimed in claim 1 , wherein the class of each diagram element is identified by classifying the one or more strokes of the hand-drawn input as text and non-text.
4. A system as claimed in claim 1 , the at least one non-transitory computer readable medium configured to:
parse the classified strokes to a handwriting recognition system for recognition of the diagram elements of the classified strokes.
5. A system as claimed in claim 4 , the at least one non-transitory computer readable medium configured to:
identify the type of each diagram element based on the recognized diagram elements and positional relationships between the diagram elements.
6. A system as claimed in claim 1 , wherein the display characteristic comprises at least one of resizing or repositioning one or more diagram elements.
7. A method for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
displaying a plurality of input diagram elements in interactive digital ink on a display device associated with the computing device, wherein the diagram elements comprise one or more of a geometric shape, a container, a connector, or text;
incrementally associating one or more of the diagram elements with one or more other of the diagram elements in accordance with a class and type of each diagram element;
identifying the class of each diagram element by classifying one or more strokes of the hand-drawn input as selected from a group consisting of text, non-text, and junk classifications for each of the one or more strokes; and
incrementally re-displaying the diagram elements, wherein a display characteristic of a diagram element is influenced based one or more received interactions with the digital ink, the one or more incremental associations, and a position of the one or more interactions relative to the incrementally associated diagram elements.
8. A method as claimed in claim 7 , wherein the strokes are grouped based on spatial and temporal information of the input.
9. A method as claimed in claim 7 , wherein the class of each diagram element is identified by classifying the one or more strokes of the hand-drawn input as text and non-text.
10. A method as claimed in claim 7 , comprising parsing the classified strokes to a handwriting recognition system for recognition of the diagram elements of the classified strokes.
11. A method as claimed in claim 10 , comprising identifying the type of each diagram element based on the recognized diagram elements and positional relationships between the diagram elements.
12. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
displaying a plurality of input diagram elements in interactive digital ink on a display device associated with the computing device, wherein the diagram elements comprise one or more of a geometric shape, a container, a connector, or text;
incrementally associating one or more of the diagram elements with one or more other of the diagram elements in accordance with a class and type of each diagram element;
identifying the class of each diagram element by classifying one or more strokes of the hand-drawn input as selected from a group consisting of text, non-text, and junk classifications for each of the one or more strokes; and
incrementally re-displaying the diagram elements, wherein a display characteristic of a diagram element is influenced based on one or more received interactions with the digital ink, the one or more incremental associations, and a position of the one or more interactions relative to the incrementally associated diagram elements.
13. A non-transitory computer readable medium as claimed in claim 12 , wherein the strokes are grouped based on spatial and temporal information of the input.
14. A non-transitory computer readable medium as claimed in claim 12 , wherein the class of each diagram element is identified by classifying the one or more strokes of the hand-drawn input as text and non-text.
15. A non-transitory computer readable medium as claimed in claim 12 , the method comprising parsing the classified strokes to a handwriting recognition system for recognition of the diagram elements of the classified strokes.
16. A non-transitory computer readable medium as claimed in claim 15 , the method comprising identifying the type of each diagram element based on the recognized diagram elements and positional relationships between the diagram elements.