Parsing an image of a visually structured document
View Patent ↗A method for automated semantic parsing of an image of a structured document includes acquiring the image of the structured document. The image of the structured document is lexed so as to associate each image element of a plurality of image elements of the image with a predefined token. A user defined template of expected semantically significant elements of the structured document is input into a parser, the expected elements being defined in a visibly pushdown language (VPL) format. The tokens are parsed into the expected elements. A computer readable medium containing executable instructions and a system are also described.
1. A method for automated semantic parsing of an image of a structured document, the method comprising:
acquiring the image of the structured document by an image acquiring device;
lexing the image of the structured document so as to associate each image element of a plurality of image elements of the image with a predefined token;
wherein lexing the image comprises associating an element of the image with a plurality of possible tokens, each association with a possible token being assigned a likelihood value;
inputting a user defined template of expected semantically significant elements of the structured document into a parser, the expected elements being defined in a visibly pushdown language (VPL) format; and
parsing one or more of the tokens into an element of the expected elements.
2. The method of claim 1 , wherein the document is a graphic user interface (GUI).
3. The method of claim 1 , wherein inputting the template comprises compiling the template by a compiler.
4. The method of claim 3 , wherein compiled output of the compiler comprises a VPL graph of the template.
5. The method of claim 1 , wherein parsing comprises execution of execution elements of a plurality of coroutines.
6. The method of claim 5 , comprising selecting for execution an execution element of a coroutine of said plurality of coroutines based on a score that is assigned to each coroutine, the score representing a likelihood of an object that was identified during execution of an execution element of that coroutine.
7. The method of claim 6 , wherein the score is calculated on the basis of a weighted product of likelihood values of sub-elements of the object.
8. A non-transitory computer readable medium having stored thereon instructions that when executed by a processor will cause the processor to perform the method of:
lexing an acquired image of a structured document so as to associate each image element of a plurality of image elements of the image with a predefined token;
wherein the lexing the image comprises associating an element of the image with a plurality of possible tokens, each association with a possible token being assigned a likelihood value;
inputting a user defined template of expected semantically significant elements of the structured document into a parser, the expected elements being defined in a visibly pushdown language (VPL) format; and
parsing one or more of the tokens into an element of the expected elements.
9. The computer readable medium of claim 8 , wherein the document is a graphic user interface (GUI).
10. The computer readable medium of claim 8 , wherein inputting the template comprises compiling the template by a compiler.
11. The computer readable medium of claim 10 , wherein compiled output of the compiler comprises a VPL graph of the template.
12. The computer readable medium of claim 8 , wherein parsing comprises execution of execution elements of a plurality of coroutines.
13. The computer readable medium of claim 12 , comprising selecting for execution an execution element of a coroutine of said plurality of coroutines based on a score that is assigned to each coroutine, the score representing a likelihood of an object that was identified during execution of an execution element of that coroutine.
14. The computer readable medium of claim 13 , wherein the score is calculated on the basis of a weighted product of likelihood values of sub-elements of the object.
15. A system comprising a processor and a computer readable medium connected to the processor, wherein the computer usable medium contains a set of instructions to:
lex an acquired image of a structured document so as to associate each image element of a plurality of image elements of the image with a predefined token;
wherein the lexing the image comprises associating an element of the image with a plurality of possible tokens, each association with a possible token being assigned a likelihood value;
receive a user defined template of expected semantically significant elements of the structured document into a parser, the expected elements being defined in a visibly pushdown language (VPL) format; and
parse one or more of the tokens into an element of the expected elements.
16. The system of claim 15 , comprising an image acquiring device for acquiring the image.
17. The system of claim 16 , wherein the image acquiring device comprises a scanner or a screen capture device.
18. The system of claim 15 , comprising a compiler for compiling the template.