Method of handling a failure in a task pipeline between a source of alerts and a security incident and event management (SIEM) system
A method of improving efficiency of processing alerts by a Security Incident & Event Management (SIEM) platform using a case creation platform, the method involves based upon contents of a configuration, creating a pipeline, comprising tasks, between a source of alerts and the SIEM platform, wherein, a task of the pipeline fails due to an exception, at least one object, including failed task input data, will be stored in an error log relating to the failure, and associated data will be stored in a task log, based upon additional content in the configuration, determining whether or not a rollback re-run can be attempted for the pipeline using at least a portion of tasks from a prior version/iteration of the pipeline; and if the rollback re-run can be attempted, executing the rollback re-run using the failed task input data.
1 . A method of improving efficiency of processing alerts by a Security Incident & Event Management (SIEM) platform using a case creation platform, the method comprising:
based upon contents of a configuration, creating a pipeline between a source of alerts and the SIEM platform;
executing a source task in the pipeline so that it retrieves alerts from the source of alerts;
providing an output of the source task to a first of at least two process tasks in the pipeline;
processing the output of the source task, using the first of the at least two process tasks and output first processed alerts to a second of the at least two process tasks;
processing the first processed alerts with the second of the at least two process tasks such that when the second of the at least two process tasks completes, an output of the second of the at least two process tasks will be provided to a sink task for publication as one or more SIEM cases to the SIEM platform; and
when the second of the at least two process tasks fails due to an exception, (i) at least one object, including failed task input data, will be stored in an error log relating to the failure, and (ii) associated data will be stored in a task log;
based upon additional content in the configuration, determining whether or not a rollback re-run can be attempted for the pipeline using at least a portion of tasks from a prior version/iteration of the pipeline; and
when the rollback re-run can be attempted, executing the rollback re-run using the failed task input data.
2 . The method of claim 1 , wherein when a result of the determining indicates that a rollback re-run cannot be attempted, the method further comprises:
mocking out the pipeline using contents from the error log and the task log.
3 . The method of claim 1 wherein, when a result of the determining indicates that a rollback re-run can be attempted, the method further comprises:
creating a renewed pipeline.
4 . The method of claim 3 , wherein the creating the renewed pipeline comprises:
rebuilding a prior version/iteration of the pipeline.
5 . The method of claim 3 , wherein the creating the renewed pipeline comprises:
over writing at least the failed task with a replacement task from a prior version/iteration.
6 . The method of claim 3 , wherein the creating the renewed pipeline comprises:
overwriting the failed task and all subsequent tasks in the pipeline with tasks from a prior version/iteration.
7 . The method of claim 3 , wherein the creating the renewed pipeline comprises:
overwriting the failed task and all prior tasks in the pipeline with tasks from a prior version/iteration.
8 . The method of claim 1 further comprising:
based upon the configuration, using the error log to modify and/or swap an ordering of the at least two process tasks in the pipeline before determining whether or not a rollback re-run can be attempted.
9 . The method of claim 8 , wherein the creating the renewed pipeline comprises:
rebuilding a prior version/iteration of the pipeline.
10 . The method of claim 8 , wherein the creating the renewed pipeline comprises:
over writing at least the failed task with a replacement task from a prior version/iteration.
11 . The method of claim 8 , wherein the creating the renewed pipeline comprises:
overwriting the failed task and all subsequent tasks in the pipeline with tasks from a prior version/iteration.
12 . The method of claim 8 , wherein the creating the renewed pipeline comprises:
overwriting the failed task and all prior tasks in the pipeline with tasks from a prior version/iteration.
13 . The method of claim 1 wherein the determining includes, identifying a type of error.
14 . The method of claim 13 , wherein the type involves one or more of: a mistake in code syntax, a missing argument, an incorrect argument, an incorrect reference, a wrong data type or incorrect logic.
15 . A method of improving efficiency of processing alerts by a Security Incident & Event Management (SIEM) platform using a case creation platform, the method comprising:
based upon contents of a configuration, creating a pipeline, comprising tasks having a source task and a sink task, between a source of alerts and the SIEM platform;
wherein, when a task of the pipeline fails due to an exception, at least one object, including failed task input data, will be stored in an error log relating to the failure, and associated data will be stored in a task log,
based upon additional content in the configuration, determining whether or not a rollback re-run can be attempted for the pipeline using at least a portion of tasks from a prior version/iteration of the pipeline; and when the rollback re-run can be attempted, executing the rollback re-run using the failed task input data.
16 . The method of claim 15 wherein, when a result of the determining indicates that a rollback re-run can be attempted, the method further comprises:
creating a renewed pipeline and executing the rollback re-run using the renewed pipeline.
17 . The method of claim 16 , wherein the creating the renewed pipeline comprises:
rebuilding a prior version/iteration of the pipeline.
18 . The method of claim 16 , wherein the creating the renewed pipeline comprises:
over writing at least the failed task with a replacement task from a prior version/iteration.
19 . The method of claim 16 , wherein the creating the renewed pipeline comprises:
overwriting the failed task and all subsequent tasks in the pipeline with tasks from a prior version/iteration.
20 . The method of claim 16 , wherein the creating the renewed pipeline comprises:
overwriting the failed task and all prior tasks in the pipeline with tasks from a prior version/iteration.