DEFAULT ARC FOR COMPRESSION OF DETERMINISTIC FINITE AUTOMATA (DFA) DATA FLOW GRAPHS WITHIN A DATA FLOW GRAPH-DRIVEN ANALYTICS PLATFORM HAVING ANALYTICS HARDWARE ACCELERATORS
An integrated circuit having a hardware-based regular expression (RegEx) engine configured to perform an analytical operation on a stream of data units. The RegEx engine receives a regular expression operation expressed as a finite automata (FA) graph having a plurality of nodes connected by directional arcs, each arc representing transitions between nodes of the FA graph based on criteria specified for the respective arc, the plurality of nodes including nodes, including a skip node, representing states in the regular expression operation. Beginning at a root node in the plurality of nodes, the RegEx engine steps through one or more nodes of the FA graph until arriving at a skip node and then skips N data units before transitioning on the default arc to another node in the graph.
1 . An integrated circuit, comprising:
memory; and
one or more hardware-based regular expression (RegEx) accelerators communicatively coupled to the memory, wherein each RegEx accelerator includes a regular expression engine configured to receive a DFA or NFA graph from the memory, and to perform a regular expression operation on a stream of data units based on the received graph, wherein the received graph includes a plurality of nodes connected by directional arcs, each arc representing transitions between nodes based on criteria specified for the respective arc, the plurality of nodes including nodes representing states in the regular expression operation,
wherein at least one of the arcs is a default arc, the default arc configured to skip a predefined number N of data units in the stream of data units before transitioning to another node, and
wherein the regular expression engine, before traversing the default arc, discards the next N data units received from the stream of data units.
2 . The integrated circuit of claim 1 , wherein each RegEx accelerator includes a counter,
wherein the engine loads the counter with the predefined number N and decrements the counter each time a data unit is received from the stream of data units.
3 . The integrated circuit of claim 1 , wherein each RegEx accelerator includes a counter, wherein the engine increments the counter each time a data unit is received from the stream of data units.
4 . The integrated circuit of claim 1 , wherein the regular expression engine is a DFA engine and the graph is a DFA graph.
5 . The integrated circuit of claim 1 , wherein the regular expression engine is an NFA engine and the graph is an NFA graph.
6 . The integrated circuit of claim 1 , wherein the received graph configures the regular expression engine of the DPU to apply at least a portion of a query predicate to the stream of data units.
7 . The integrated circuit of claim 1 , wherein the nodes include a hash node, wherein the regular expression engine, on encountering the hash node, compares one or more known symbols associated with the hash node to a current data unit received from the stream of data units and transitions, on a match, via an arc associated with the matched known symbol,
wherein the engine traverses the default arc when the current data unit does not match any of the known symbols, the engine reading and discarding N data units from the stream of data units before traversing the default arc.
8 . The integrated circuit of claim 7 , wherein each RegEx accelerator includes a counter, wherein the engine loads the counter with the predefined number N and decrements the counter each time a data unit is received from the stream of data units.
9 . The integrated circuit of claim 7 , wherein the regular expression engine is a DFA engine and the graph is a DFA graph.
10 . The integrated circuit of claim 7 , wherein the regular expression engine is an NFA engine and the graph is an NFA graph.
11 . The integrated circuit of claim 1 , wherein the integrated circuit further comprises one or more programmable processor cores, wherein the programmable processor cores are communicatively coupled to the memory.
12 . The integrated circuit of claim 1 , wherein the integrated circuit further comprises a cluster manager and one or more programmable processor cores, wherein the programmable processor cores are communicatively coupled to the cluster manager and to the memory.
13 . A method, comprising:
receiving a regular expression operation to be performed on a stream of data units;
identifying N data units in the stream of data units;
expressing the regular expression operation as a graph, the graph having a plurality of nodes connected by directional arcs, each arc representing transitions between states based on criteria specified for the respective arc, the plurality of nodes including nodes representing states in the regular expression operation, at least one node having a default arc, the default arc configured to skip a predefined number N of data units in the stream of data units before transitioning to another node;
beginning at a root node in the plurality of nodes, stepping a regular expression engine through one or more nodes of the graph until arriving at the node having the default arc; and
skipping the N data units before traversing the default arc.
14 . The method of claim 13 , wherein receiving the regular expression operation includes receiving a data flow graph.
15 . The method of claim 13 , wherein N is a length in data units of a portion of a payload in the stream of data units.
16 . The method of claim 13 , wherein skipping includes decrementing a counter each time one of the N data units is received from the stream of data units.
17 . The method of claim 13 , wherein the plurality of nodes further includes a second node having a second default arc, the second default arc configured to skip a predefined number M of data units in the stream of data units before traversing the second default arc.
18 . In an integrated circuit having a memory and a regular expression engine, a method comprising:
receiving a graph at the regular expression engine, the graph including a plurality of nodes connected by directional arcs, each arc representing transitions between nodes based on criteria specified for the respective arc, the plurality of nodes including nodes representing states in the regular expression operation, at least one node having a default arc;
beginning at a root node in the plurality of nodes, stepping the regular expression engine through one or more nodes of the graph until arriving at the node having the default arc; and
skipping N data units in a stream of data units before traversing the default arc.
19 . The method of claim 18 , wherein skipping N data units includes:
loading the predefined number N to a counter;
reading and discarding data units from the stream of data units, decrementing the counter each time a data unit is read; and
traversing the default arc when the counter=0.
20 . The method of claim 18 , wherein the plurality of nodes includes a node having a second default arc, the second default arc configured to skip a predefined number M of data units in the stream of data units before traversing the second default arc.