Disjunctive rule mining with finite automaton hardware
The present invention introduces the development of a flexible CPU-AP (Computer Processing Unit-Automata Processor) computing infrastructure for mining hierarchical patterns based on Apriori algorithm. A novel automaton design strategy, called linear design, is described to generate automata for matching and counting hierarchical patterns and apply it on SPM (Sequential Pattern Mining). In addition, another novel automaton design strategy, called reduction design, is described for the disjunctive rule matching (DRM) and counting. The present invention shows performance improvement of AP SPM and DRM solutions and broader capability over multicore and GPU (Graphics Processing Unit) implementations of GSP SPM, and shows that AP SPM and DRM solutions outperform state-of-the-art SPM algorithms SPADE and PrefixSpan (especially for larger datasets).
1. A processor for mining a plurality of patterns in large datasets, the processor comprises a plurality of functional element comprising:
a plurality of state transition elements;
a plurality of counters; and
a plurality of Boolean elements,
wherein the processor is configured to implement non-deterministic finite automata (NFA) with a reduction design method,
wherein the reduction design method is used for disjunctive rule mining (DRM) with an Apriori-based algorithm, and
wherein the disjunctive rule mining comprises the steps of:
generating candidates; and
matching and counting d-rules,
wherein in the DRM, a d-rule is defined as a set of disjunctive items (d-items), y i ={d s1 , d s2 , . . . , d sl }.
2. The processor according to claim 1 , wherein the matching and counting d-rules is performed by a d-rule matching automaton of the reduction design method, the d-rule matching automaton comprising:
a plurality of starters;
a plurality of d-item sub-structures; and
a plurality of AND sub-structures,
wherein the plurality of starters activate the plurality of d-item sub-structures during an input transaction, the plurality of d-item sub-structures match the d-items, and the plurality of AND sub-structures do AND operation on all of the plurality of d-item sub-structures within the same automaton.
3. The processor according to claim 2 , wherein each of the plurality of starters comprises two state transition elements, a first state transition element of each of the plurality of starters represents the beginning of a new input transition and activates all d-item sub-structures when matches a transaction delimiter, and a second state transition element of each of the plurality of starters matches any valid item and keeps activating all d-item sub-structures before the end of the current transaction,
wherein each of the plurality of d-item sub-structures comprises three state transition elements, a first state transition element of each of the plurality of d-item sub-structures holds a set of items in one d-item, a second state transition element of each of the plurality of d-item sub-structures holds an activated status of the d-item sub-structure until the end of the current transaction, and a third state transition element of each of the plurality of d-item sub-structures waits until the end of the current transaction to finish d-item matching, and
wherein all outputs of the plurality of d-item sub-structures connect to an AND unit of the plurality of AND sub-structures.
4. The processor according to claim 2 , wherein the d-rule matching automaton is pre-compiled and loaded in the runtime.
5. The processor according to claim 2 , wherein bit-wise parallelism of the plurality of state transition elements is used to represent an OR relation of the alternative items in the d-item.
6. The processor according to claim 2 , wherein the plurality of Boolean elements are utilized to calculate an AND relation among the d-items of the d-rule.
7. The processor according to claim 1 , wherein the reduction design method does not require a sorting operation.
8. A method for mining a plurality of patterns in large datasets by a processor, the method comprising the steps of:
implementing non-deterministic finite automata (NFA) with a reduction design method on the processor, the processor comprising:
a plurality of state transition elements;
a plurality of counters; and
a plurality of Boolean elements,
wherein the reduction design method is used for disjunctive rule mining (DRM) with an Apriori-based algorithm, and
wherein the disjunctive rule mining comprises the steps of:
generating candidates; and
matching and counting d-rules,
wherein in the DRM, a d-rule is defined as a set of disjunctive items (d-items), y i ={d s1 , d s2 , . . . , d sl }.
9. The method according to claim 8 , wherein the matching and counting d-rules is performed by a d-rule matching automaton of the reduction design method, the d-rule matching automaton comprising:
a plurality of starters;
a plurality of d-item sub-structures; and
a plurality of AND sub-structures,
wherein the plurality of starters activate the plurality of d-item sub-structures during an input transaction, the plurality of d-item sub-structures match the d-items, and the plurality of AND sub-structures do AND operation on all of the plurality of d-item sub-structures within the same automaton.
10. The method according to claim 9 , wherein each of the plurality of starters comprises two state transition elements, a first state transition element of each of the plurality of starters represents the beginning of a new input transition and activates all d-item sub-structures when matches a transaction delimiter, and a second state transition element of each of the plurality of starters matches any valid item and keeps activating all d-item sub-structures before the end of the current transaction,
wherein each of the plurality of d-item sub-structures comprises three state transition elements, a first state transition element of each of the plurality of d-item sub-structures holds a set of items in one d-item, a second state transition element of each of the plurality of d-item sub-structures holds an activated status of the d-item sub-structure until the end of the current transaction, and a third state transition element of each of the plurality of d-item sub-structures waits until the end of the current transaction to finish d-item matching, and
wherein all outputs of the plurality of d-item sub-structures connect to an AND unit of the plurality of AND sub-structures.
11. The method according to claim 9 , wherein the d-rule matching automaton is pre-compiled and loaded in the runtime.
12. The method according to claim 9 , wherein bit-wise parallelism of the plurality of state transition elements is used to represent an OR relation of the alternative items in the d-item.
13. The method according to claim 9 , wherein the plurality of Boolean elements are utilized to calculate an AND relation among the d-items of the d-rule.
14. The method according to claim 8 , wherein the reduction design method does not require a sorting operation.