Accurate and transparent path prediction using process mining
The present disclosure generally relates to the field of data structures and in particular, a loop-aware footprint matrix data structure adapted for data process traversal. The proposed approach is directed to a computer-based analytic system and corresponding method that uses a specific data structure and processing thereof, in some embodiments, adapted to computationally estimate predictions of next events by first generating a data structure based on business process models obtained using process mining techniques, and then using the improved data structure for generating predictions, which can then be encapsulated in the form of computer instructions or machine instruction sets, having a specific sequence for execution.
1 . A computer implemented method for maintaining a matrix data structure adapted for storing an electronic representation of a data process log including one or more data process traces, each data process trace representing a sequential series of execution instructions, the method comprising:
receiving, at a processor, a process tree data structure generated by a process discovery engine configured to process the data process log to record, in a computer memory, algebraic splits represented in the data process log as one or more data operator functions;
initializing, at the processor, the matrix data structure by:
instantiating, in the computer memory, one or more rows, each row corresponding to a corresponding data process trace in the one or more data process traces;
instantiating, in the computer memory, one or more columns, each column corresponding to a corresponding data operator function in the one or more data operator functions, wherein the one or more data operator functions comprises at least one loop element;
populating, at the processor, data corresponding to each cell of the matrix data structure by replaying the corresponding data process trace on the process tree data structure to determine one or more indices assigned to each data operator function of the one or more data operator functions, wherein the one or more indices comprises information relating to an execution frequency of the at least one loop element;
wherein:
the process discovery engine is configured to recursively process the data process log to record algebraic splits;
the data process log represents an execution log of executed computer instructions by a computer processor, and each data process trace of the one or more data process traces represents a set of sequential processing events of the computer processor; and
the process discovery engine includes a plurality of process discovery mechanisms used in concert to generate a plurality of process path predictions;
training the process discovery engine using a machine learning engine and a training set of labelled paths and corresponding predictions to determine an optimal process discovery mechanism of the plurality of process discovery mechanisms; and
tuning the process discovery engine to utilize the optimal process discovery mechanism of the plurality of process discovery mechanisms.
2 . The method of claim 1 , wherein the each sequential processing event represents a transition between states of a concurrent processing model maintained by the computer processor.
3 . The method of claim 1 , wherein the process discovery engine is an inductive data miner engine.
4 . The method of claim 1 , wherein the training set of labelled paths includes decomposed segmentations of the one or more data process traces, where the one or more data process traces are decomposed into a plurality of combinations of prefixes and suffixes, such that the prefixes establish an evaluation set and the suffixes establish a ground truth set.
5 . The method of claim 1 , wherein the one or more data process traces are clustered into a plurality of clusters grouping similar data process traces; wherein a number of clusters of the plurality of clusters is determined using a hyperparameter optimization of a type grid search using a portion of a training data set and wherein the clustering is conducted using a soft clustering approach where, for each data process trace, a probability of the data process trace belonging to each cluster in the plurality of clusters is established.
6 . A computer implemented method for maintaining a matrix data structure adapted for storing a representation of a data process log, wherein the data process log represents an execution log of executed computer instructions by a computer processor, including one or more data process traces, each data process trace representing a sequential series of execution instructions and a set of sequential processing events of the computer processor, the method comprising:
receiving, at a processor, a process tree data structure generated by a process discovery engine configured to recursively process the data process log to record algebraic splits represented in the data process log as one or more data operator functions;
initializing, at the processor, the matrix data structure by:
instantiating one or more rows, each row corresponding to a corresponding data process trace in the one or more data process traces;
instantiating one or more columns, each column corresponding to a corresponding data operator function in the one or more data operator functions; and
populating, at the processor, each cell of the matrix data structure by replaying the corresponding data process trace on the process tree data structure to determine one or more indices assigned to each data operator function of the one or more data operator functions; and
recursively generating, at the processor, a process path prediction suffix for a prefix representing a set of n events observed from an uncompleted data process trace using the matrix data structure by:
iteratively adding sequential events to the process path prediction suffix until an end of a Petri Net represented in the process tree data structure by:
generating, a list of active tokens;
establishing, from the list of active tokens, a list of active transitions;
while a number of active transitions is greater than one, recursively:
selecting a selected data operator function of the one or more data operator functions that is common to at least two transitions and that is closest to a root;
selecting a predicted transition depending on an operator type of the selected data operator function, the predicted transition including at least one of a selection of a branch for a next execution, a decision to be established at an exclusive gateway, or whether to stay in or leave a loop;
determining an updated number of active transitions, and if the number of active transitions is greater than one, recursing to the selecting of a next data operator function;
executing the predicted transition to add a sequential event onto the process path prediction suffix; and
returning the suffix as a data structure, which in combination with the prefix represents a predicted completed data process trace based on the uncompleted data process trace.
7 . The method of claim 6 , wherein one or more execution processing instructions for a computer processor are generated based on the predicted completed data process trace.
8 . The method of claim 7 , wherein:
the selecting of the predicted transition includes determining that a particular order established in a current prefix being recursed is not represented in any of the one or more data process traces; and the selecting of the predicted transition further includes a first, a second, and a third sequential step of selection, which are applied consecutively when a previous step fails;
a first sequential step is to use the matrix data structure as-is;
a second sequential step is to drop a loop portion of the matrix data structure and to concatenate columns for a same operator; and
a third sequential step is to make a decision by observing only the Petri Net represented by the process tree data structure.
9 . A computer implemented method for maintaining a matrix data structure adapted for storing a representation of a data process log, wherein the data process log represents an execution log of executed computer instructions by a computer processor, including one or more data process traces, each data process trace representing a sequential series of execution instructions and a set of sequential processing events of the computer processor, the method comprising:
receiving, at a processor, a process tree data structure generated by a process discovery engine configured to recursively process the data process log to record algebraic splits represented in the data process log as one or more data operator functions;
initializing, at the processor, the matrix data structure by:
instantiating one or more rows, each row corresponding to a corresponding data process trace in the one or more data process traces;
instantiating one or more columns, each column corresponding to a corresponding data operator function in the one or more data operator functions; and
populating, at the processor, each cell of the matrix data structure by replaying the corresponding data process trace on the process tree data structure to determine one or more indices assigned to each data operator function of the one or more data operator functions;
wherein:
the one or more data process traces are clustered into a plurality of clusters grouping similar data process traces, the clustering is conducted using a soft clustering approach where, for each data process trace, a probability of the data process trace belonging to each cluster in the plurality of clusters is established;
a number of clusters of the plurality of clusters is determined using a hyperparameter optimization of a type grid search using a portion of a training data set;
only the similar data process traces having probabilities greater than a pre-defined value of belonging to each cluster of the plurality of clusters are used to establish the process tree data structure; and
the process tree data structure is transformed to a Petri Net such that the similar data process traces having probabilities less than or equal to the pre-defined value can be replayed upon the process tree data structure.
10 . The method of claim 9 wherein a stochastic gradient descent classifier is trained to predict which cluster a prefix belongs to, and a suffix of a given prefix is predicted using the cluster returned by the stochastic gradient descent classifier.
11 . A computer implemented system for maintaining a matrix data structure adapted for storing an electronic representation of a data process log including one or more data process traces, each data process trace representing a sequential series of execution instructions, the system comprising:
a process discovery engine configured to generate a process tree data structure by processing the data process log to record, in computer memory, algebraic splits represented in the data process log as one or more data operator functions;
a computer processor configured to initialize the matrix data structure on data storage by:
instantiating in the computer memory one or more rows, each row corresponding to a corresponding data process trace in the one or more data process traces;
instantiating in the computer memory one or more columns, each column corresponding to a corresponding data operator function in one or more data operator functions, wherein the one or more data operator functions comprises at least one loop element; and
populating data corresponding to each cell of the matrix data structure by replaying the corresponding data process trace on the process tree data structure to determine one or more indices assigned to each data operator function of the one or more data operator functions, wherein the one or more indices comprises information relating to an execution frequency of the at least one loop element;
wherein:
the process discovery engine is further configured to recursively process the data process log to record algebraic splits; and
the process discovery engine includes a plurality of process discovery mechanisms used in concert to generate a plurality of process path predictions; and the process discovery engine is adapted to:
train the process discovery engine using a machine learning engine and a training set of labelled paths and corresponding predictions to determine an optimal process discovery mechanism of the plurality of process discovery mechanisms; and
tune the process discovery engine utilize the optimal process discovery mechanism of the plurality of process discovery mechanisms;
wherein the training set of labelled paths includes decomposed segmentations of the one or more data process traces, where the one or more data process traces are decomposed into a plurality of combinations of prefixes and suffixes, such that the prefixes establish an evaluation set and the suffixes establish a ground truth set.
12 . A computer implemented system for maintaining a matrix data structure adapted for storing a representation of a data process log including one or more data process traces, each data process trace representing a sequential series of execution instructions, the system comprising:
a process discovery engine configured to generate a process tree data structure by recursively processing the data process log to record algebraic splits represented in the data process log as one or more data operator functions; and
a computer processor configured to:
initialize the matrix data structure on data storage by:
instantiating one or more rows, each row corresponding to a corresponding data process trace in the one or more data process traces;
instantiating one or more columns, each column corresponding to a corresponding data operator function in one or more data operator functions; and
populating each cell of the matrix data structure by replaying the corresponding data process trace on the process tree data structure to determine one or more indices assigned to each data operator function of the one or more data operator functions; and
recursively generate a process path prediction suffix for a prefix representing a set of n events observed from an uncompleted data process trace using the matrix data structure by:
iteratively adding sequential events to the process path prediction suffix until an end of a Petri Net represented in the process tree data structure by:
generating, a list of active tokens;
establishing, from the list of active tokens, a list of active transitions;
while a number of active transitions is greater than one, recursively:
selecting a selected data operator function of the one or more data operator functions that is common to at least two transitions and that is closest to a root;
selecting a predicted transition depending on a type of the selected data operator function, the predicted transition including at least one of a selection of a branch for a next execution, a decision to be established at an exclusive gateway, or whether to stay in or leave a loop;
determining an updated number of active transitions, and if the number of active transitions is greater than one, recursing to the selecting of a next data operator function;
executing the predicted transition to add a sequential event onto the process path prediction suffix; and
returning the suffix as a data structure, which in combination with the prefix represents a predicted completed data process trace based on the uncompleted data process trace.
13 . The system of claim 12 , wherein the one or more data process traces are clustered into a plurality of clusters grouping similar data process traces; wherein a number of clusters of the plurality of clusters is determined using a hyperparameter optimization of a type grid search using a portion of a training data set and wherein the clustering is conducted using a soft clustering approach where, for each data process trace, a probabilities of the data process trace belonging to each cluster of the plurality of clusters is established.
14 . The system of claim 13 , wherein only the similar data process traces having probabilities greater than a pre-defined value of belonging each cluster of the plurality of clusters are used to establish the process tree data structure, and the process tree data structure is transformed to the Petri Net such that the similar data process traces having probabilities less than or equal to the pre-defined value can be replayed upon the process tree data structure; and wherein a stochastic gradient descent classifier is trained to predict which cluster the prefix belongs to, and a suffix of a given prefix is predicted using the cluster returned by the stochastic gradient descent classifier.