Multi-engine intrusion detection system
Example methods and systems for multi-engine intrusion detection are described. In one example, a computer system may configure a set of multiple intrusion detection system (IDS) engines that include at least a first IDS engine and a second IDS engine. In response to detecting establishment of a first packet flow and a second packet flow, the computer system may assign the first packet flow to the first IDS engine and second packet flow to the second engine based on an assignment policy. This way, first packet flow inspection may be performed using the first IDS engine to determine whether first packet(s) associated with the first packet flow are potentially malicious. Second packet flow inspection may be performed using the second IDS engine to determine whether second packet(s) associated with the second packet flow are potentially malicious.
1 . A method for a computer system to perform multi-engine intrusion detection, wherein the method comprises:
configuring a set of multiple intrusion detection system (IDS) engines that include at least a first IDS engine and a second IDS engine;
in response to detecting an establishment of a first packet flow associated with an endpoint, assigning the first packet flow to the first IDS engine based on an assignment policy;
in response to detecting an establishment of a second packet flow associated with the endpoint, assigning the second packet flow to the second IDS engine based on the assignment policy;
performing, using the first IDS engine, first packet flow inspection to determine whether one or more first packets associated with the first packet flow are potentially malicious based on a set of IDS signatures; and
performing, using the second IDS engine, second packet flow inspection to determine whether one or more second packets associated with the second packet flow are potentially malicious based on the set of IDS signatures,
wherein configuring the set of multiple IDS engines comprises at least one of the following:
configuring the first IDS engine to include (a) a first queue to store one or more first requests associated with the first packet flow inspection, and (b) a first worker thread to process the one or more first requests; and
configuring the second IDS engine to include (a) a second queue to store one or more second requests associated with the second packet flow inspection, and (b) a second worker thread to process the one or more second requests, and
wherein performing the first packet flow inspection using the first IDS engine comprises:
acquiring a first lock associated with the first queue that stores the one or more first requests associated with the first packet flow;
generating a first snapshot associated with the first queue, wherein the first snapshot stores the one or more first requests;
releasing the first lock associated with the first queue; and
processing the one or more first requests in the first snapshot in a substantially lock-free manner.
2 . The method of claim 1 , further comprising at least one of the following:
in response to detecting the one or more first packets associated with the first packet flow, the network kernel driver acquiring a first lock associated with the first queue to queue the one or more first requests in the first queue; and
in response to detecting the one or more second packets associated with the second packet flow, the network kernel driver acquiring a second lock associated with the second queue to queue the one or more second requests in the second queue.
3 . The method of claim 1 , wherein the method further comprises:
monitoring metric information associated with the set of multiple IDS engines; and
based at least on the metric information, performing dynamic scaling of the set of multiple IDS engines to increase or decrease a size of the set of multiple IDS engines.
4 . The method of claim 3 , wherein performing dynamic scaling comprises at least one of the following:
in response to determination that a scale-up condition is satisfied based on the metric information and system resource availability, increasing the size by adding at least one further IDS engine to the set of multiple IDS engines; and
in response to determination that a scale-down condition is satisfied based on the metric information and system resource availability, decreasing the size by removing at least one of the multiple IDS engines from the set.
5 . The method of claim 1 , wherein assigning the first packet flow to the first IDS engine or the second packet flow to the second IDS engine comprises:
retrieving the assignment policy that is configured based on one of the following: round robin policy, load-based policy, application-affinity-based policy, and priority-based policy.
6 . The method of claim 1 , wherein processing the one or more first requests in the substantially lock-free manner comprises processing the one or more first requests from the first snapshot while additional requests associated with the first packet flow are queued in the first queue.
7 . A non-transitory computer-readable storage medium that includes a set of instructions which, in response to execution by a processor of a computer system, cause the processor to perform a method of multi-engine intrusion detection, wherein the method comprises:
configuring a set of multiple intrusion detection system (IDS) engines that include at least a first IDS engine and a second IDS engine;
in response to detecting an establishment of a first packet flow associated with an endpoint, assigning the first packet flow to the first IDS engine based on an assignment policy;
in response to detecting an establishment of a second packet flow associated with the endpoint, assigning the second packet flow to the second IDS engine based on the assignment policy;
performing, using the first IDS engine, first packet flow inspection to determine whether one or more first packets associated with the first packet flow are potentially malicious based on a set of IDS signatures; and
performing, using the second IDS engine, second packet flow inspection to determine whether one or more second packets associated with the second packet flow are potentially malicious based on the set of IDS signatures,
wherein configuring the set of multiple IDS engines comprises at least one of the following:
configuring the first IDS engine to include (a) a first queue to store one or more first requests associated with the first packet flow inspection, and (b) a first worker thread to process the one or more first requests; and
configuring the second IDS engine to include (a) a second queue to store one or more second requests associated with the second packet flow inspection, and (b) a second worker thread to process the one or more second requests, and
wherein performing the first packet flow inspection using the first IDS engine comprises:
acquiring a first lock associated with the first queue that stores the one or more first requests associated with the first packet flow;
generating a first snapshot associated with the first queue, wherein the first snapshot stores the one or more first requests;
releasing the first lock associated with the first queue; and
processing the one or more first requests in the first snapshot in a substantially lock-free manner.
8 . The non-transitory computer-readable storage medium of claim 7 , wherein the method further comprises at least one of the following:
in response to detecting the one or more first packets associated with the first packet flow, a network kernel driver acquiring a first lock associated with the first queue to queue the one or more first requests in the first queue; and
in response to detecting the one or more second packets associated with the second packet flow, the network kernel driver acquiring a second lock associated with the second queue to queue the one or more second requests in the second queue.
9 . The non-transitory computer-readable storage medium of claim 7 , wherein the method further comprises:
monitoring metric information associated with the set of multiple IDS engines; and
based at least on the metric information, performing dynamic scaling of the set of multiple IDS engines to increase or decrease a size of the set of multiple IDS engines.
10 . The non-transitory computer-readable storage medium of claim 9 , wherein performing dynamic scaling comprises at least one of the following:
in response to determination that a scale-up condition is satisfied based on the metric information and system resource availability, increasing the size by adding at least one further IDS engine to the set of multiple IDS engines; and
in response to determination that a scale-down condition is satisfied based on the metric information and system resource availability, decreasing the size by removing at least one of the multiple IDS engines from the set.
11 . The non-transitory computer-readable storage medium of claim 7 , wherein assigning the first packet flow to the first IDS engine or the second packet flow to the second IDS engine comprises:
retrieving the assignment policy that is configured based on one of the following: round robin policy, load-based policy, application-affinity-based policy, and priority-based policy.
12 . The non-transitory computer-readable storage medium of claim 7 , wherein processing the one or more first requests in the substantially lock-free manner comprises processing the one or more first requests from the first snapshot while additional requests associated with the first packet flow are queued in the first queue.
13 . A computer system capable of performing multi-engine intrusion detection that includes at least a first intrusion detection system (IDS) engine and a second IDS engine, wherein the computer system is configured to:
in response to detecting an establishment of a first packet flow associated with an endpoint, assign the first packet flow to a first IDS engine based on an assignment policy; and
in response to detecting an establishment of a second packet flow associated with the endpoint, assign the second packet flow to a second IDS engine based on the assignment policy;
perform first packet flow inspection using the first IDS engine to determine whether one or more first packets associated with the first packet flow are potentially malicious based on a set of IDS signatures; and
perform second packet flow inspection using the second IDS engine to determine whether one or more second packets associated with the second packet flow are potentially malicious based on the set of IDS signatures, wherein
the first IDS engine is configured to include (a) a first queue to store one or more first requests associated with the first packet flow inspection, and (b) a first worker thread to process the one or more first requests, and the second IDS engine is configured to include (a) a second queue to store one or more second requests associated with the second packet flow inspection, and (b) a second worker thread to process the one or more second requests, and
the first IDS engine is configured to perform the first packet flow inspection by performing the following:
acquiring a first lock associated with the first queue that stores the one or more first requests associated with the first packet flow;
generating a first snapshot associated with the first queue, wherein the first snapshot stores the one or more first requests;
releasing the first lock associated with the first queue; and
processing the one or more first requests in the first snapshot in a substantially lock-free manner.
14 . The computer system of claim 13 , wherein the computer system further comprises a network kernel driver configured to:
in response to detecting the one or more first packets associated with the first packet flow, acquire a first lock associated with the first queue to queue the one or more first requests in the first queue; and
in response to detecting the one or more second packets associated with the second packet flow, acquire a second lock associated with the second queue to queue the one or more second requests in the second queue.
15 . The computer system of claim 13 , wherein the computer system is further configured to:
monitor metric information associated with a set of multiple IDS engines including the first and second IDS engines; and
based at least on the metric information, perform dynamic scaling of the set of multiple IDS engines to increase or decrease a size of the set of multiple IDS engines.
16 . The computer system of claim 15 , wherein the computer system is configured to perform dynamic scaling by performing at least one of the following:
in response to determination that a scale-up condition is satisfied based on the metric information and system resource availability, adding at least one further IDS engine to the set of multiple IDS engines; and
in response to determination that a scale-down condition is satisfied based on the metric information and system resource availability, removing at least one of the multiple IDS engines from the set.
17 . The computer system of claim 13 , wherein the computer system is configured to assign the first packet flow to the first IDS engine or the second packet flow to the second IDS engine by performing the following:
retrieving the assignment policy that is configured based on one of the following: round robin policy, load-based policy, application-affinity-based policy, and priority-based policy.
18 . The computer system of claim 13 , wherein processing the one or more first requests in the substantially lock-free manner comprises processing the one or more first requests from the first snapshot while additional requests associated with the first packet flow are queued in the first queue.