System and method for detecting patterns in structured fields of network traffic packets
A computer method and system for determining patterns in network traffic packets having structured subfields for generating filter candidate regular expressions for DDoS attack mitigation. Stored packets are analyzed to extract a query name for each stored packet. Each query name is segregated into subfields. A Results-table is generated utilizing the segregated subfields of the query names. A Field-length table is generated that contains the length of the Field Values (Field-length) for each Field Name and an associated counter indicating how many instances the Field-length for a Field Name is present in the extracted query names. The Field-length table is analyzed to determine patterns of equal length in the “Results” table. Utilizing the Patterns table, unique combinations of the Field Values are generated as a filter candidate regular expression for DDoS attack mitigation purposes.
1 . A method comprising:
analyzing stored packets to extract a query name for each stored packet;
segregating each query name into subfields;
generating a results table utilizing the segregated subfields of the query names, wherein the results table includes a field name associated with a field value and counter value indicating how many instances the associated field value is present in the associated field name for each extracted query name;
generating, utilizing the results table, a field length table that contains the length of the field values as a field length for each field name and an associated counter indicating how many instances the field length for a field name is present in the extracted query names;
sorting the results table and the field length table based on respective counter values for individual field values;
eliminating field name entries from the results table that do not exceed a threshold value of the number of extracted query names to generate a patterns table;
analyzing the field length table to determine, for each field and field length, patterns of equal length in the results table; and
generating, utilizing the patterns table, unique combinations of the field values, wherein generating unique combinations of the field values comprises generating candidate regular expressions, evaluating the candidate regular expressions across all packets containing DNS query names, and accepting candidate regular expressions which match more than the threshold value, wherein each generated unique combination is a filter candidate regular expression for DDOS attack mitigation purposes; and
blocking a network according to the generated filter candidate regular expression.
2 . The method of claim 1 , wherein the unique combinations of the field values are generated in reverse order relative to a query name.
3 . The method of claim 1 , further comprising: in response to analyzing the field length table, summing the counter values, and subtracting the summed counter values from a total count of Domain Name system (DNS) packets.
4 . The method of claim 1 , wherein the network traffic packets include DNS packets.
5 . The method of claim 4 , wherein the DDOS attack is one of a reflection attack or amplification attack.
6 . The method of claim 4 , wherein the DDOS attack is a DNS water torture attack.
7 . The method of claim 1 , further including:
intercepting network traffic packets flowing from one or more external computers to the one or more protected computer networks; and
storing the intercepted packets in a storage device.
8 . The method of claim 7 , wherein the DNS packets include DNS queries and DNS query responses.
9 . The method of claim 8 , wherein the storage device contains Packet Capture (PCAP) files.
10 . The method of claim 9 , wherein a predetermined number of DNS responses are stored in the PCAP.
11 . A computer system comprising:
one or more databases having memory configured to store instructions;
a processor disposed in communication with the memory, wherein the processor upon execution of the instructions is configured to:
analyze stored packets to extract a query name for each stored packet;
segregate each query name into subfields;
generating a results table utilizing the segregated subfields of the query names, wherein the results table includes a field name associated with a field value and counter value indicating how many instances the associated field value is present in the associated field name for each extracted query name;
generate, utilizing the results table, a field length table that contains the length of the field values, wherein generating unique combinations of the field values comprises generating candidate regular expressions, evaluating the candidate regular expressions across all packets containing DNS query names, and accepting candidate regular expressions which match more than the threshold value, as a field length for each field name and an associated counter indicating how many instances the field length for a field name is present in the extracted query names;
sort the results table and the field length table based on respective counter values for individual field values;
eliminate field name entries from the results table that do not exceed a threshold value of the number of extracted query names to generate a patterns table;
analyze the field length table to determine, for each field and field length, patterns of equal length in the results table; and
generate, utilizing the patterns table, unique combinations of the field values wherein each generated unique combination is a filter candidate regular expression for DDOS attack mitigation purposes; and
block a network according to the generated filter candidate regular expression.
12 . The computer system as recited in claim 11 , wherein the network traffic packets include DNS packets.
13 . The computer system as recited in claim 12 , wherein the DDOS attack is one of a reflection attack or amplification attack.
14 . The computer system as recited in claim 12 , wherein the DDOS attack is a DNS Water Torture attack.
15 . The computer system as recited in claim 11 , wherein the processor is further configured to:
intercept network traffic packets flowing from one or more external computers to the one or more protected computer networks; and
store the intercepted packets in a storage device.
16 . The computer system as recited in claim 15 , wherein the DNS packets include DNS queries and DNS query responses.
17 . The computer system as recited in claim 16 , wherein the storage device contains Packet Capture (PCAP) files.
18 . The computer system as recited in claim 17 , wherein a predetermined number of DNS responses are stored in the PCAP.
19 . A non-transitory computer-readable media comprising computer-executable instructions embodied thereon that, when executed by a processor, cause the processor to perform a process comprising:
analyzing stored packets to extract a query name for each stored packet;
segregating each query name into subfields;
generating a results table utilizing the segregated subfields of the query names, wherein the results table includes a field name associated with a field value and counter value indicating how many instances the associated field value is present in the associated field name for each extracted query name;
generating, utilizing the results table, a field length table that contains the length of the field values as a field length for each field name and an associated counter indicating how many instances the field length for a field name is present in the extracted query names;
sorting the results table and the field length table based on respective counter values for individual field values;
eliminating field name entries from the results table that do not exceed a threshold value of the number of extracted query names to generate a patterns table;
analyzing the field length table to determine, for each field and field length, patterns of equal length in the results table; and
generating, utilizing the patterns table, unique combinations of the field values, wherein generating unique combinations of the field values comprises generating candidate regular expressions, evaluating the candidate regular expressions across all packets containing DNS query names, and accepting candidate regular expressions which match more than the threshold value, wherein each generated unique combination is a filter candidate regular expression for DDOS attack mitigation purposes; and
blocking a network according to the generated filter candidate regular expression.
20 . The non-transitory computer-readable media of claim 19 , wherein the unique combinations of the field values are generated in reverse order relative to a query name.
21 . The method of claim 1 , wherein segregating each query name into subfields comprises segregating each query name based upon a full stop character “.” in each query name starting at the top-level domain (TLD) of each query name whereby a first subfield corresponds to a TLD of each query name.
22 . The method of claim 1 , wherein analyzing the field length table to determine patterns of equal length comprises: adding together counter values for patterns of equal length in the results table to generate a sum value, subtracting the sum value from a counter for a corresponding field-length table entry to generate a remainder value, and inserting the field-length value and counter to the patterns table when the remainder value exceeds the threshold value.
23 . The method of claim 1 , wherein the method further comprises: after analyzing all entries in the field length table, adding together all counter values for each field-length, subtracting the added counter values from a total count of captured DNS packets, and inserting a catch-all entry containing the value ‘*’ for a field into the patterns table when a remainder value exceeds a threshold value.
24 . The method of claim 1 , further comprising removing regular expressions which match legitimate user traffic to reduce risk of overblocking legitimate requests.