Dynamic lexer object construction
In one example in accordance with the present disclosure, a method for dynamic lexer object construction includes receiving a grammar rule including a variable and a Boolean function to be performed on a character sequence using the variable. The method includes receiving the character sequence and a value corresponding to the character sequence and constructing a lexer object that includes the value as the variable used by the Boolean function. The method includes parsing the character sequence using the lexer object.
1. A method, comprising:
receiving, as an input, a value for a variable of a grammar rule, the grammar rule defining a structure of a character sequence to be parsed and including a default value for the variable;
replacing the default value in the grammar rule with the received value;
receiving the character sequence;
executing a lexer object that includes a Boolean function that evaluates the character sequence based on the grammar rule; and
parsing the character sequence using the lexer object.
2. The method of claim 1 , wherein the Boolean function includes a matching function and parsing the character sequence further comprises matching the value to a token in the character sequence.
3. The method of claim 1 , wherein the Boolean function is an unmatching function and parsing the character sequence further comprises identifying a token in the character sequence that does not match the variable and determining that the identified token includes a word character.
4. The method of claim 1 , wherein the variable is specified via an application programming interface.
5. The method of claim 1 , wherein the variable represents a standardized value.
6. The method of claim 1 , wherein the function is added to the grammar rule as an extension of a lexer class.
7. A system comprising:
a processor to:
receive an extension of a lexer class defining a Boolean function and a variable used by the Boolean function, wherein the variable represents a value to be found in a character sequence;
replace a default value in a grammar rule with the variable;
receiver to receive the character sequence;
execute a lexer object that includes the Boolean function, wherein the Boolean function evaluates the character sequence based on the grammar rule; and
parse the character sequence using the lexer object.
8. The system of claim 7 , wherein the value is one of: a token, a separator, a character set or a wordchar.
9. The system of claim 7 , wherein the Boolean function includes a matching function and the parser further matches the value to a token in the character sequence.
10. The system of claim 7 , wherein the Boolean function is an unmatching function and the parser further identifies a token in the character sequence that does not match the variable and determines that the identified token includes a word character.
11. The system of claim 7 , wherein the lexer object is to further:
construct the lexer object at run time without re-compiling a grammar rule.
12. A non-transitory machine-readable storage medium comprising instructions that when executed by a processor are to cause the processor to:
receive a grammar rule including a value and a Boolean function to be performed on a character sequence, wherein the Boolean function uses a default value for parsing the character sequence;
receive the character sequence;
construct a lexer object based on the received value; and
parse the character sequence using the lexer object.
13. The non-transitory machine-readable storage medium of claim 12 , wherein the received value represents data to be found in a character sequence.
14. The non-transitory machine-readable storage medium of claim 12 , wherein the received value is specified via an application programming interface.
15. The non-transitory machine-readable storage medium of claim 12 , further comprising instructions that are to cause the processor to:
construct the lexer object in a lexer level.
16. The method of claim 1 , wherein the Boolean function includes a matching function, and wherein parsing the character sequence comprises matching a token of the character sequence with the received value.
17. The method of claim 16 , wherein the value comprises a separator, the method further comprising:
generating a parsed array comprising a plurality of tokens that were separated by the separator in the character sequence.
18. The method of claim 16 , wherein the value comprises a replacement character set, the method further comprising:
replacing a first character set of tokens in the character sequence based on the replacement character set.
19. The method of claim 16 , wherein the value comprises a replacement word, the method further comprising:
replacing a first word in the character sequence based on the replacement word.
20. The method of claim 1 , wherein receiving the value comprises:
retrieving, at runtime, the value from a configuration file that encodes the value.