NON-CONSTANT DATA ENCODING FOR TABLE-DRIVEN SYSTEMS
Parse tables or like representations are augmented with extension points to enable call out to arbitrary code. Such parse tables can be automatically generated from a specification including fixed information along with information about extensibility points provided. The extensibility points enable incorporation of dynamic data into a fixed parse table. In one instance, this allows a parser to determine if a character is acceptable at the time of execution rather than when the parse table was defined.
1 . A parser generation system, comprising:
an interface component that receives a lexical specification; and
a generator component that produces an extensible parse map based on the specification, the map includes fixed data that identifies state transitions as a function of input and current state and one or more extension points to enable arbitrary code invocation.
2 . The system of claim 1 , the lexical specification identifies the one or more extension points that identify the arbitrary code.
3 . The system of claim 2 , the extension points includes one or more special delimiters and reference to the code.
4 . The system of claim 2 , the lexical specification includes variable definitions and employment of defined variables.
5 . The system of claim 1 , the arbitrary code corresponds to an alternate scanner.
6 . The system of claim 1 , the map is a table.
7 . The system of claim 6 , the table specifies the arbitrary code with a character in a reserved range.
8 . The system of claim 6 , the table references an index that identifies the arbitrary code.
9 . The system of claim 1 , the arbitrary code determines if a token should be allowed based on rules at the time of execution.
10 . The system of claim 1 , the map is a finite function.
11 . The system of claim 1 , further comprising a component that compresses the map.
12 . A parser generation method, comprising:
acquiring a lexical specification including an extension point; and
generating a parse table that comprises a set of fixed data identifying state transitions based on current input and state and an extension point that specifies external code to facilitate identification of state.
13 . The method of claim 12 , producing code that employs the parse table to guide parsing of a programmatic language.
14 . The method of claim 12 , further comprising denoting the extension point with a character from a restricted range character set.
15 . The method of claim 12 , further comprising generating the external code.
16 . The method of claim 15 , further comprising referencing another parse table to facilitate parsing of an embedded language.
17 . The method of claim 12 , further comprising compressing the parse table into a compact and efficient representation.
18 . A computer-readable medium having stored thereon a parse table, comprising:
a number of columns identifying input characters; and
a number of rows identifying parsing states, the intersection between the columns and rows identifies either a state transition or an extensibility point that calls out to arbitrary code, the parse table includes at least one extensibility point.
19 . The computer-readable medium of claim 18 , the extensibility point is encoded as a character from a reserved character range.
20 . The computer-readable medium of claim 19 , the character identifies particular code or an index from which the code can be located.