Methods and systems for efficient packet filtering
A packet gateway may protect TCP/IP networks by enforcing security policies on in-transit packets that are crossing network boundaries. The policies may include packet filtering rules derived from cyber threat intelligence (CTI). The rapid growth in the volume of CTI and in the size of associated CTI-derived policies, coupled with ever-increasing network link speeds and network traffic volume, may cause the costs of sufficient computational resources to be prohibitive. To efficiently process packets, a packet gateway may be provided with at least one probabilistic data structure, such as a Bloom filter, for testing packets to determine if packet data may match a packet filtering rule. Packet filtering rules may be grouped into subsets of rules, and a data structure may be provided for determining a matching subset of rules associated with a particular packet.
1. A method comprising:
receiving, by a packet gateway located at a boundary between a protected network and an unprotected network, a plurality of packets;
determining, for each packet of the plurality of packets, at least one packet matching criterion associated with the packet;
testing, for each packet of the plurality of packets, at least one policy probabilistic data structure for the at least one packet matching criterion;
based on a determination that a first packet of the plurality of packets does not match at least one packet matching criterion of the at least one policy probabilistic data structure, forwarding the first packet towards its intended destination;
based on a determination that a second packet of the plurality of packets matches at least one packet matching criterion associated with the at least one policy probabilistic data structure, determining at least one of a plurality of policy subset probabilistic data structures associated with at least one packet matching criterion of the second packet;
testing a determined at least one of the plurality of policy subset probabilistic data structures associated with at least one packet matching criterion of the second packet; and
based on a determination that the second packet of the plurality of packets matches at least one packet matching criterion associated with the determined at least one of the plurality of policy subset probabilistic data structures, performing a rule action associated with the determined at least one of the plurality of policy subset probabilistic data structures.
2. The method of claim 1 , wherein the at least one policy probabilistic data structure and each of the policy subset probabilistic data structures are Bloom filters or Cuckoo filters.
3. The method of claim 1 , wherein the at least one policy probabilistic data structure has a higher false positive rate than any of the policy subset probabilistic data structures.
4. The method of claim 1 , wherein a policy subset probabilistic data structure associated with an action to prevent packet transmission has a lower false positive rate than a policy subset probabilistic data structure associated with an action to allow packet transmission to proceed.
5. The method of claim 1 , further comprising:
receiving, by the packet gateway, a plurality of packet filtering rules, wherein each of the packet filtering rules comprises at least one packet matching criterion;
generating at least one policy probabilistic data structure representing the plurality of packet filtering rules;
partitioning the plurality of packet filtering rules into a plurality of rule subsets, wherein each of the plurality of rule subsets is associated with a common rule action; and
generating a plurality of policy subset probabilistic data structures, wherein each of the plurality of policy subset probabilistic data structures is associated with one of the plurality of rule subsets, wherein each of the policy subset probabilistic data structures is associated with the common rule action associated with associated rule subset.
6. The method of claim 5 , wherein generating a plurality of policy subset probabilistic data structures comprises:
partitioning the plurality of packet filtering rules based on an associated common packet matching criterion type to determine common packet matching criterion type rule groups;
partitioning each of the common packet matching criterion type rule groups based on an associated common rule action to determine the rule subsets; and
generating a policy subset probabilistic data structure corresponding to each rule subset with the associated common rule action and the associated common packet matching criterion type.
7. The method of claim 1 , wherein testing a determined at least one of the plurality of policy subset probabilistic data structures associated with at least one packet matching criterion of the second packet comprises:
determining a plurality of packet matching criterion types associated with the second packet;
determining at least one subset probabilistic data structure corresponding to the determined plurality of packet matching criterion types associated with the second packet; and
testing each subset probabilistic data structure corresponding to the determined plurality of packet matching criterion types until a match is determined.
8. The method of claim 1 , further comprising:
receiving, by the packet gateway, at least one new rule, wherein the at least one new rule comprises at least one new packet matching criterion;
updating the at least one policy probabilistic data structure to represent the at least one new rule;
determining, by the packet gateway, a rule subset to be updated based on the at least one new packet matching criterion; and
updating a policy subset probabilistic data structure corresponding to the rule subset to be updated based on the at least one new packet matching criterion of the at least one new rule.
9. The method of claim 1 , wherein generating the plurality of policy subset probabilistic data structures comprises applying an indicator encoding algorithm to each of a plurality of packet matching criteria associated with each rule subset to populate a subset probabilistic data structure corresponding to the rule subset.
10. A method comprising:
receiving, by a packet gateway located at a boundary between a protected network and an unprotected network, a plurality of packets;
testing, by the packet gateway and for each packet of the plurality of packets, at least one policy probabilistic data structure representing a security policy to determine whether each packet of the plurality of packets is associated with at least one rule of the security policy, wherein the security policy comprises a plurality of packet filtering rules;
based on a determination that a first packet of the plurality of packets matches at least one packet matching criterion associated with the at least one policy probabilistic data structure, determining at least one of a plurality of policy subset probabilistic data structures;
testing, for the first packet, the at least one of the plurality of policy subset probabilistic data structures; and
based on the testing the at least one of the plurality of policy subset probabilistic data structures, filtering the first packet.
11. The method of claim 10 , further comprising:
based on a determination that a second packet of the plurality of packets does not match the at least one packet matching criterion associated with the at least one policy probabilistic data structure, forwarding the second packet to its intended destination.
12. The method of claim 10 , wherein filtering the first packet comprises:
performing a rule action on the first packet.
13. The method of claim 10 , wherein filtering the first packet comprises:
searching a rule set associated with the determined at least one of the plurality of policy subset probabilistic data structures; and
performing, based on the rule set, a rule action.
14. The method of claim 10 , further comprising:
based on a determination that a second packet of the plurality of packets does not match at least one second packet matching criterion associated with at least one of the plurality of policy subset probabilistic data structures, forwarding the second packet to its intended destination.
15. The method of claim 10 , wherein filtering the first packet comprises one of blocking or monitoring the first packet.
16. The method of claim 10 , further comprising:
receiving, by the packet gateway, at least one new rule, wherein the at least one new rule comprises at least one corresponding packet matching criterion; and
updating the at least one policy probabilistic data structure to represent the at least one new rule.
17. The method of claim 10 , wherein testing the at least one policy probabilistic data structure comprises using an encryption key to test an encoded policy probabilistic data structure.