Efficient DFA generation for non-matching characters and character classes in regular expressions
A character class is detected in a regular expression and substituted with a pseudo character. A table is created with a bit vector for each pseudo character inserted into the regular expression. Each bit in the bit-vector represents one character of the alphabet from which the expression is generated. The status of the bits in a bit-vector indicates which characters of the alphabet are included in the character class. The pseudo character in the modified regular expression is used to construct a non-deterministic finite automaton (NFA). The NFA with the pseudo character is then used to construct a deterministic finite automaton (DFA). When constructing the DFA, the bit-vectors are used to determine if a certain transition should be constructed in the DFA.
1. A method for generating a deterministic finite automaton (DFA) via a processor, the method comprising:
receiving an expression associated with malware, wherein the received expression is to be constructed into a DFA;
executing instructions stored in memory, the instructions executed by a processor to:
detect a character class in the received expression;
substitute a pseudo character for the detected character class in the received expression;
generate for the substituted pseudo character, a corresponding bit-vector, wherein the generated bit-vector includes a bit for each of a plurality of alphabet characters, wherein a status of a respective bit within the generated bit-vector reflects whether a respective alphabet character is included in the detected character class of the received expression;
update a table in memory regarding the received expression, wherein the updated table stores the received expression in association with the substituted pseudo character and the generated bit-vector for the detected character class of the received expression;
construct a non-deterministic finite automaton (NFA) from the received expression based on the updated table, wherein constructing the NFA includes inserting the substituted pseudo character in place of the detected character class within the received expression;
construct a DFA from the constructed NFA, wherein constructing the DFA from the constructed NFA includes:
retrieving the generated bit-vector associated with the substituted pseudo character included in the NFA, the generated bit-vector retrieved from the updated table; and
constructing transitions for each node of the NFA based on the retrieved bit-vector;
compares the DFA constructed from the NFA with the stored information in the updated table for the received expression to identify whether the DFA matches the stored information for the received expression; and
identifies that the received expression includes malware when the DFA matches the stored information for the received expression; and
sending a communication to a user device, the communication including an alert that the received expression includes the malware.
2. The method of claim 1 , further comprising:
parsing the received expression; and
detecting a second character class in the parsed expression.
3. The method of claim 1 , further comprising creating the table and storing the table in memory.
4. The method of claim 3 , further comprising:
detecting that the received expression includes a different character class associated with a different pseudo character; and
generating a different bit-vector for the different character class detected in the expression.
5. The method of claim 4 , further comprising updating the table regarding the different bit-vector for the different character class detected in the received expression.
6. The method of claim 1 , wherein the expression is received from an appliance administrator.
7. The method of claim 1 , wherein the expression includes a beginning character and a closing character common to a plurality of expressions.
8. A non-transitory computer-readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for generating a definite finite automaton (DFA), the method comprising:
receiving an expression associated with malware, wherein the received expression is to be constructed into a DFA;
detecting a character class in the received expression;
substituting a pseudo character for the detected character class in the received expression;
generating for the substituted pseudo character, a corresponding bit-vector, wherein the generated bit-vector include a bit for each of a plurality of alphabet characters and a status of a respective bit within the generated bit-vector reflects whether a respective alphabet character is included in the detected character class of the received expression;
updating a table in memory regarding the received expression, wherein the updated table stores the received expression in association with the substituted pseudo character and the generated bit-vector for the detected character class of the received expression;
constructing a non-deterministic finite automaton (NFA) from the received expression having at least one substituted pseudo character;
constructing a DFA from the constructed NFA, wherein constructing the DFA from the constructed NFA includes:
retrieving the generated bit-vector associated with the substituted pseudo character included in the NFA, the generated bit-vector retrieved from the updated table, and
constructing transitions for each node of the NFA based on the retrieved bit-vector;
comparing the DFA constructed from the NFA with the stored information in the updated table for the received expression to identify whether the DFA matches the stored information for the received expression; and
identifying that the received expression includes malware when the DFA matches the stored information for the received expression; and
sending a communication to a user device, the communication including an alert that the received expression includes the malware.
9. The non-transitory computer readable storage medium of claim 8 , wherein the program further comprises instructions executable to:
parse the received expression; and
detect a character class in the parsed expression.
10. The non-transitory computer readable storage medium of claim 8 , wherein the program further comprises instructions executable to create the table and store the table in memory.
11. The non-transitory computer readable storage medium of claim 10 , the program further executable to:
detect that the received expression includes a different character class associated with a different pseudo character; and
generate a different bit-vector for the different character class detected in the expression.
12. The non-transitory computer readable storage medium of claim 11 , the program further executable to update the table regarding the different bit-vector for the different character class detected in the received expression.
13. The non-transitory computer-readable storage medium of claim 8 , wherein the expression is received from an appliance administrator.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the expression includes a beginning character and a closing character common to a plurality of expressions.
15. A system for generating a definite finite automaton (DFA), the system comprising:
a memory;
a processor that executes an application is stored in the memory, wherein the processor executes the application to:
receive an expression associated with malware, wherein the received expression is to be constructed into a DFA,
detect a character class in the received expression,
substitute a pseudo character for the detected character class in the received expression;
generate for the substituted pseudo character, a corresponding bit-vector; wherein the generated bit-vector includes a bit for each of a plurality of alphabet characters, wherein a status of a respective bit identifying within the generated bit-vector reflects whether a respective alphabet character is included in the detected character class of the received expression;
update a table in memory regarding the received expression, wherein the updated table stores the received expression in association with the substituted pseudo character and the generated bit-vector for the detected character class of the received expression;
construct a non-deterministic finite automaton (NFA) from the received expression substituted based on the updated table, wherein constructing the NFA includes inserting the pseudo character in place of the detected character class within the received expression;
construct a DFA from the NFA, wherein constructing the DFA includes:
retrieving the generated bit-vector associated with the substituted pseudo character included in the NFA, the generated bit-vector retrieved from the updated table; and
constructing transitions for each node of the NFA based on the retrieved bit-vector;
compares the DFA constructed from the NFA with the stored information in the updated table for the received expression to identify whether the DFA matches the stored information for the received expression; and
identifies that the received expression includes malware when the DFA matches the stored information for the received expression; and
sending a communication to a user device, the communication including an alert that the received expression includes the malware.
16. The system of claim 15 , wherein the processor further executes the application to parse the received expression and detect a second character class in the parsed expression.
17. The system of claim 15 , wherein the processor further executes the application to create the table and to store the table in the memory.
18. The system of claim 17 , wherein the processor further executes instructions to:
detect that the received expression includes a different character class associated with a different pseudo character; and
generate a different bit-vector for the different character class detected in the expression.
19. The system of claim 18 , wherein the process further executes instructions to update the table regarding the different bit-vector for the different character class detected in the received expression.
20. The system of claim 15 , further comprising a user interface that receives the expression from an appliance administrator.