Incremental DFA compilation with single rule granularity
View Patent ↗A composite DFA for multiple regular expressions or other rules may be generated in a two-step process—first compiling single rule DFAs, then performing subset construction on those DFAs to generate the composite DFA, with subset information retained. A new batch of one or more rules may be added by another subset construction from the old composite DFA and new single rule DFAs, with subset information for the new composite DFA compressed into sets of states from old and new single rule DFAs. A batch of one or more rules is deleted by deleting references to single rule DFA states from composite DFA subsets, deleting composite DFA states with empty subsets and merging composite DFA states with identical subsets. Rules are changed by deleting the old versions and then adding the new versions.
1. A method of constructing a composite DFA (deterministic finite automaton) using subset construction, said method comprising:
compiling at least one single-rule DFA;
performing a first subset construction on said at least one single-rule DFA to generate a first composite DFA, retaining subset information for said first composite DFA;
compiling at least one new rule into at least one corresponding additional single-rule DFA;
performing a second subset construction to generate a second composite DFA, wherein said first composite DFA acts as a first NFA (non-deterministic finite automaton) input for said second subset construction and said at least one corresponding additional single-rule DFA acts as at least one additional NFA input for said second subset construction, retaining subset information for said second composite DFA;
searching said subset information for subsets containing single-rule DFA states corresponding to a rule to be deleted;
deleting all single-rule DFA state references found during the search from said subset information; and
deleting any composite DFA state whose subset becomes empty after said step of deleting all single-rule DFA state references.
2. The method of claim 1 , wherein said at least one new rule comprises a plurality of new rules, and said at least one corresponding additional single-rule DFA comprises a plurality of corresponding single-rule DFAs.
3. The method of claim 1 , said method further comprising merging said composite DFA states whose subsets become identical after the step of deleting all single-rule DFA state references.
4. The method of claim 1 , further comprising deleting all token IDs within said composite DFA which indicate matches to said deleted rule.
5. A method of constructing a composite DFA (deterministic finite automaton) using subset construction, said method comprising:
compiling a plurality single-rule DFAs;
performing a first subset construction on said plurality single-rule DFAs to generate a first composite DFA, retaining subset for said first composite DFA;
searching said subset information for subsets containing single-rule DFA states corresponding to a rule to be deleted;
deleting all single-rule DFA state references found during said search from said subset information; and
deleting any composite DFA state whose subset becomes empty after said step of deleting all single-rule DFA state references.
6. The method of claim 5 , further comprising deleting all token IDs within said composite DFA which indicate matches to said deleted rule.
7. The method of claim 5 , further comprising merging the composite DFA states whose subsets become identical after the step of deleting all single-rule DFA state references.
8. A computer system comprising a non-transitory computer readable medium including programmed instructions, a compiler and a first set of rules for constructing a composite DFA (deterministic finite automaton) using subset construction, wherein the instructions, when executed by a computer, cause the computer system to:
compile a first set of single-rule DFAs from said first set of rules;
subset construct said single-rule DFAs into a first composite DFA, retaining subset information for said first composite DFA;
compile at least one new rule into at least one corresponding additional single-rule DFA;
subset construct a second composite DFA using said first composite DFA as a first NFA (non-deterministic finite automaton) input for subset construction and said at least one corresponding additional single-rule DFA as at least one additional NFA input for subset construction, retaining subset information;
delete at least one rule from said set of rules by use of a search of said subset information for subsets containing single-rule DFA states corresponding to a rule to be deleted;
delete all single rule DFA state references found during said search from said subset information; and
delete any composite DFA state whose subset becomes empty.
9. The system of claim 8 , wherein said computer system is further caused to delete all token IDs within said composite DFA which indicate matches to the deleted at least one rule.
10. The system of claim 8 , wherein the computer system is further caused to merge said composite DFA states whose subsets become identical.