Systems and methods for process mining using unsupervised learning and for automating orchestration of workflows
A system for discovering business processes using unsupervised learning is configured to: (a) receive multimodal event data from a plurality of sources, the multimodal event data including a plurality of event instances; (b) associate the multimodal event data with a vector representation, such that the plurality of event instances is represented as a plurality of event vectors; (c) correlate the plurality of event vectors using unsupervised learning to identify one or more processes; and (d) generate a process model script for the one or more processes. A method for automated orchestration of a workflow is also disclosed.
1 . A system for discovering business processes using unsupervised learning, the system including a non-transitory computer-readable medium storing computer-executable instructions thereon such that when the instructions are executed, the system is configured to:
receive multimodal event data from a plurality of sources associated with at least a first user and a second user, the multimodal event data including a plurality of event instances, the plurality of event instances including (i) keystroke activity for when a key was pressed and application telemetry associated with a software application in which the key presses were inputted and (ii) screen captures;
associate the multimodal event data with a vector representation, such that the plurality of event instances is represented as a plurality of event vectors;
correlate, via a neural network, the plurality of event vectors using unsupervised learning to identify one or more processes, wherein a first process of the one or more processes requires multimodal event data from the first user on a first device and the second user on a second device, the first device and the second device being separate devices;
refine the one or more processes by:
continuing to monitor, in real-time, the plurality of sources for more multimodal event data,
determining that the multimodal event data received in real-time belongs to a second process in the one or more processes that is different from the first process,
determining errors associated with the first process and the second process, and
updating weights of the neural network based on the errors associated with the first process and the second process; and
generate a process model script for the one or more processes, wherein the generated process model script includes graphical probability indications in the one or more processes.
2 . The system of claim 1 , further configured to correlate the plurality of event vectors by:
joining a first subset of the plurality of event vectors to create a first process matrix,
joining a second subset of the plurality of event vectors to create a second process matrix,
determining a similarity between the first process matrix and the second process matrix, the similarity measured as a dot product between the first process matrix and the second process matrix, and
identifying that the first process matrix and the second process matrix refer to a same process in the one or more processes based on the similarity being below a threshold.
3 . The system of claim 1 , further configured to correlate the plurality of event vectors by:
joining a first subset of the plurality of event vectors to create a first process matrix,
joining a second subset of the plurality of event vectors to create a second process matrix,
determining a similarity between the first process matrix and the second process matrix, the similarity measured as a dot product between the first process matrix and the second process matrix, and
identifying that the first process matrix and the second process matrix are different processes in the one or more processes based on the similarity being above a threshold.
4 . The system of claim 1 , further configured to correlate the plurality of event vectors using a long short term memory (LSTM) neural network.
5 . The system of claim 1 , wherein the process model script includes one or more directed graphs.
6 . The system of claim 1 , wherein the process model script is a robotic process automation (RPA) script.
7 . The system of claim 1 , wherein the plurality of sources includes two or more selected from the group consisting of: one or more Internet Information Services (IIS) log files, one or more Apache log file, one or more application log files, one or more standard operating procedure (SOP) manuals, one or more screen capture logs, one or more keystroke logs, one or more business process documents (BPDs).
8 . The system of claim 1 , further configured to apply an image contained in the screen captures to a VGG-16 convolutional neural network to obtain an image embedding as the vector representation of the image.
9 . A method for discovering business processes using unsupervised learning, the method comprising:
receiving multimodal event data from a plurality of sources associated with at least a first user and a second user, the multimodal event data including a plurality of event instances, the plurality of event instances including (i) keystroke activity for when a key was pressed and application telemetry associated with a software application in which the key presses were inputted and (ii) screen captures;
associating the multimodal event data with a vector representation, such that the plurality of event instances is represented as a plurality of event vectors;
correlating, via a neural network, the plurality of event vectors using unsupervised learning to identify one or more processes, wherein a first process of the one or more processes requires multimodal event data from the first user on a first device and the second user on a second device, the first device and the second device being separate devices;
refining the one or more processes by:
continuing to monitor, in real-time, the plurality of sources for more multimodal event data,
determining that the multimodal event data received in real-time belongs to a second process in the one or more processes that is different from the first process,
determining errors associated with the first process and the second process, and
updating weights of the neural network based on the errors associated with the first process and the second process; and
generating a process model script for the one or more processes, wherein the generated process model script includes graphical probability indications in the one or more processes.
10 . The method of claim 9 , wherein correlate the plurality of event vectors comprises:
joining a first subset of the plurality of event vectors to create a first process matrix,
joining a second subset of the plurality of event vectors to create a second process matrix,
determining a similarity between the first process matrix and the second process matrix, the similarity measured as a dot product between the first process matrix and the second process matrix, and
identifying that the first process matrix and the second process matrix refer to a same process in the one or more processes based on the similarity being below a threshold.
11 . The method of claim 9 , wherein correlating the plurality of event vectors comprises:
joining a first subset of the plurality of event vectors to create a first process matrix,
joining a second subset of the plurality of event vectors to create a second process matrix,
determining a similarity between the first process matrix and the second process matrix, the similarity measured as a dot product between the first process matrix and the second process matrix, and
identifying that the first process matrix and the second process matrix are different processes in the one or more processes based on the similarity being above a threshold.
12 . The method of claim 9 , wherein correlating the plurality of event vectors is performed using a long short term memory (LSTM) neural network.
13 . The method of claim 9 , wherein the process model script includes one or more directed graphs.
14 . A computer-implemented method of orchestrating a workflow, comprising the steps of:
receiving multimodal event data from a plurality of sources associated with at least a first user and a second user, the multimodal event data including a plurality of event instances, the plurality of event instances including (i) keystroke activity for when a key was pressed and application telemetry associated with a software application in which the key presses were inputted and (ii) screen captures;
associating the multimodal event data with a vector representation, such that the plurality of event instances is represented as a plurality of event vectors;
correlating, via a neural network, the plurality of event vectors using unsupervised learning to identify one or more processes, wherein a first process of the one or more processes requires multimodal event data from the first user on a first device and the second user on a second device, the first device and the second device being separate devices;
refining the one or more processes by:
continuing to monitor, in real-time, the plurality of sources for more multimodal event data,
determining that the multimodal event data received in real-time belongs to a second process in the one or more processes that is different from the first process,
determining errors associated with the first process and the second process, and
updating weights of the neural network based on the errors associated with the first process and the second process;
generating a process model script for the one or more processes, wherein the generated process model script includes graphical probability indications in the one or more processes;
storing the one or more processes as a plurality of micro actions in a software library, wherein each of the micro actions represents a computer program or operation programmed from at least a partially interpreted programming language and embedded in or invoked from a markup language, wherein the markup language includes a data serialization language, has a structure permitting storage of multiple documents in a single file or stream, and can automatically expand a reference to an environmental variable to allow one or more additional micro actions to be embedded in or invoked from the markup language, and wherein at least some of the micro actions are reusable across multiple digital transformation domains;
presenting at least a subset of the micro actions as distinct selectable graphical elements on a graphical palette element displayed on an electronic display;
receiving, via a human-machine interface device, a selection of a selected one of the distinct selectable graphical elements, which is dragged and dropped onto a design workspace displayed on the electronic display, wherein the selected one of the distinct selectable graphical elements corresponds to a selected one of the plurality of micro actions, the selected micro action including:
a plurality of definable attributes defining one or more input parameters for the selected micro action,
a script or program to be executed or invoked by the markup language,
an outcome of the execution returned from the script or program executed or invoked by the markup language, and
a configurable navigation option that specifies at least a normal navigation flow or an alternate flow based on the outcome to at least one other of the plurality of micro actions;
connecting using the configurable navigation option the selected one of the plurality of micro actions with at least the at least one other of the plurality of micro actions to generate a workflow, and responsive to an instruction to publish the workflow, automatically generating human-readable code in the markup language, the human-readable code including the selected one of the plurality of micro actions and the at least one other of the plurality of micro actions; and
storing the generated workflow in the markup language to orchestrate the generated workflow as an automated process.
15 . The method of claim 14 , wherein correlate the plurality of event vectors comprises:
joining a first subset of the plurality of event vectors to create a first process matrix,
joining a second subset of the plurality of event vectors to create a second process matrix,
determining a similarity between the first process matrix and the second process matrix, the similarity measured as a dot product between the first process matrix and the second process matrix, and
identifying that the first process matrix and the second process matrix refer to a same process in the one or more processes based on the similarity being below a threshold.
16 . The method of claim 14 , wherein correlating the plurality of event vectors comprises:
joining a first subset of the plurality of event vectors to create a first process matrix,
joining a second subset of the plurality of event vectors to create a second process matrix,
determining a similarity between the first process matrix and the second process matrix, the similarity measured as a dot product between the first process matrix and the second process matrix, and
identifying that the first process matrix and the second process matrix are different processes in the one or more processes based on the similarity being above a threshold.
17 . The method of claim 14 , wherein the outcome includes one or more of a set of values or a json or xml document, the method further comprising:
labeling the selected one of the micro actions to identify a reason for incomplete process automation, including a failure of a remote service, a lack of response from a human-in-the-loop, or a failure of one or more internal components.
18 . The method of claim 14 , further comprising receiving the instruction to publish the workflow, and responsive thereto, publishing the workflow by retrieving metadata associated with the stored workflow and automatically generating the human-readable code in the markup language without any human intervention or input.
19 . The method of claim 18 , wherein the publishing the workflow includes parsing a json script or an xml document associated with the stored workflow and extracting metadata from the workflow during the generating the markup language.
20 . The method of claim 14 , wherein the screen captures include an image, and wherein associating the multimodal event data with the vector representation includes applying the image to a VGG-16 convolutional neural network to obtain an image embedding as the vector representation of the image.