Systems and methods for network security memory reduction via distributed rulesets
The present disclosure describes systems and methods for reducing rule set sizes via statistical redistribution throughout a plurality of network security appliances. A rule set may be generated for each security appliance that includes (i) a first set of rules based on known attacks, identified as rules for mandatory inclusion in the rule set; and (ii) a subset of the second set of rules, identified as rules for potential inclusion in the rule set, selected randomly according to a distribution percentage, score, or weight for each potentially included rule. Higher scored rules, which may be more likely vectors for potential attack, may be distributed to a greater number of appliances; while lower scored rules that may be less likely or represent more speculative attacks may be distributed to fewer appliances.
1. A method for providing distributed rule sets for network security appliances, comprising:
generating, by a management server, from a rule database comprising a plurality of packet processing rules, a first rule set for a first security appliance by:
adding, to the first rule set, a first subset of the packet processing rules consisting of packet processing rules identified in the rule database as mandatory,
determining that a storage size of the first rule set is below a threshold, and
adding, to the first rule set, a second subset of the packet processing rules consisting of packet processing rules not identified in the rule database as mandatory, responsive to the determination that the storage size of the first rule set is below the threshold;
generating, by the management server, a second rule set for a second security appliance by:
adding, to the second rule set, the first subset of the packet processing rules,
determining that a storage size of the second rule set is below the threshold, and
adding, to the second rule set, a third subset of the packet processing rules consisting of packet processing rules not identified in the rule database as mandatory, responsive to the determination that the storage size of the second rule set is below the threshold and responsive to addition of the second subset of the packet processing rules to the first rule set, the third subset different from the second subset; and
transmitting, by the management server, the first rule set to the first security appliance and the second rule set to the second security appliance.
2. The method of claim 1 , wherein each packet processing rule is associated with a distribution percentage, and wherein the first subset consists of one or more packet processing rules associated with a 100% distribution percentage.
3. The method of claim 2 , wherein the second subset and third subset each consist of one or more packet processing rules associated with distribution percentages less than 100%.
4. The method of claim 1 , wherein adding the first additional packet processing rule to the first rule set further comprises:
selecting the first additional packet processing rule;
generating a random distribution score value; and
adding the first additional packet processing rule to the first rule set responsive to the random distribution score value being less than a distribution percentage associated with the first additional packet processing rule.
5. The method of claim 4 , wherein generating the random distribution score value further comprises retrieving the random distribution score value from a random number generator or pseudo random number generator of the management server.
6. The method of claim 4 , wherein generating the random distribution score value further comprises calculating a result of a hash function with inputs comprising a device identifier of the first security appliance.
7. The method of claim 1 , further comprising adding an association of a device identifier of the first security appliance, in the rule database, to each entry corresponding to each packet processing rule included in the first rule set.
8. The method of claim 1 , wherein generating the first rule set further comprises determining that the storage size of the first rule set exceeds the predetermined threshold; and wherein transmitting the first rule set to the first security appliance is performed responsive to the determination.
9. A system for providing distributed rule sets for network security appliances, comprising:
a management server comprising a processor executing a rule distributor, a network interface in communication with a first security appliance and a second security appliance, and a storage device comprising a rule database comprising a plurality of packet processing rules comprising a first subset of packet processing rules identified as mandatory and a plurality of additional packet processing rules;
wherein the rule distributor is configured to:
generate a first rule set for a first security appliance by:
adding, to the first rule set, a first subset of the packet processing rules consisting of packet processing rules identified in the rule database as mandatory,
determining that a storage size of the first rule set is below a threshold, and
adding, to the first rule set, a second subset of the packet processing rules consisting of packet processing rules not identified in the rule database as mandatory, responsive to the determination that the storage size of the first rule set is below the threshold, and
generate a second rule set for a second security appliance by:
adding, to the second rule set, the first subset of the packet processing rules,
determining that a storage size of the second rule set is below the threshold, and
adding, to the second rule set, a third subset of the packet processing rules consisting of packet processing rules not identified in the rule database as mandatory, responsive to the determination that the storage size of the second rule set is below the threshold and responsive to addition of the second subset of the packet processing rules to the first rule set, the third subset different from the second subset; and
wherein the network interface is configured to transmit the first rule set to the first security appliance and the second rule set to the second security appliance.
10. The system of claim 9 , wherein each packet processing rule is associated with a distribution percentage, and wherein the first subset consists of one or more packet processing rules associated with a 100% distribution percentage.
11. The system of claim 10 , wherein the second subset and third subset each consist of one or more packet processing rules associated with distribution percentages less than 100%.
12. The system of claim 9 , wherein the rule distributor is further configured for:
selecting the first additional packet processing rule,
generating a random distribution score value, and
adding the first additional packet processing rule to the first rule set responsive to the random distribution score value being less than a distribution percentage associated with the first additional packet processing rule.
13. The system of claim 12 , wherein the management server further comprises a random number generator or pseudo random number generator; and wherein the rule distributor is further configured for retrieving the random distribution score value from the random number generator or pseudo random number generator.
14. The system of claim 12 , wherein the rule distributor is further configured for calculating a result of a hash function with inputs comprising a device identifier of the first security appliance.
15. The system of claim 9 , wherein the rule distributor is further configured for adding an association of a device identifier of the first security appliance, in the rule database, to each entry corresponding to each packet processing rule included in the first rule set.
16. The system of claim 9 , wherein the rule distributor is further configured for determining that the storage size of the first rule set exceeds the predetermined threshold; and wherein transmitting the first rule set to the first security appliance is performed responsive to the determination.