CREATING AGGREGATE NETWORK FLOW TIME SERIES IN NETWORK ANOMALY DETECTION SYSTEMS

1 . A computer implemented method for processing network flow data over a time series associated with a device-circuit pair, comprising:

receiving, by a server, flow data for one or more flows that correspond to the device-circuit pair;

calculating, by the server, a time difference for each flow of the one or more flows that correspond to the device-circuit pair;

based on the calculated time differences and the received flow data, updating, by the server, a probability distribution model associated with the device-circuit pair; and

determining, by the server, whether a time bucket, of the time series, is complete or open based on the updated probability distribution model.

2 . The method of claim 1 , further comprising:

in response to determining that the time bucket is determined to be complete, ignoring, by the server, further flow data that corresponds to the time bucket; and

in response to determining that the time bucket is determined to be open, incorporating, by the server, further flow data that corresponds to the time bucket.

3 . The method of claim 2 , further comprising:

in response to determining that the time bucket is determined to be complete, sending, by the server, flow data that corresponds to the time bucket to a detection module to detect possible network anomalies.

4 . The method of claim 1 , wherein the probability distribution model comprises flow data that corresponds to the device-circuit pair and time differences for flows that correspond to the device-circuit pair.

5 . The method of claim 4 , wherein the updating the probability distribution model comprises:

incorporating, by the server, the received flow data and the calculated time differences into the probability distribution model;

calculating, by the server, a mean value based on the time differences and the flow data included in the probability distribution model; and

calculating, by the server, a standard deviation value based the time differences and the flow data included in the probability distribution model.

6 . The method of claim 5 , wherein the determining whether the time bucket is complete or open comprises:

calculating, by the server, a time delay value based on the standard deviation value; and

determining, by the server, whether the time bucket is complete or open based on the time delay value and a file stamp time value of a network flow record containing the received flow data.

7 . The method of claim 6 , wherein the calculating the time delay value comprises calculating, by the server, the time delay value based on the standard deviation value and the mean value.

8 . The method of claim 6 , wherein the determining whether the time bucket is complete or open comprises:

creating, by the server, an expiry time based on an end time of the time bucket and the calculated time delay value;

determining, by the server, that the time bucket is complete if the file stamp time is beyond the created expiry time; and

determining, by the server, that the time bucket is open if the file stamp time is not beyond the created expiry time.

9 . The method of claim 4 , wherein each of the time differences in the probability distribution model is a time difference between a start time of each flow in the probability distribution model and a file stamp time of a corresponding network flow record.

10 . A system for processing network flow data over a time series associated with a device-circuit pair, comprising:

a memory; and

at least one processor coupled to the memory and configured to:

receive flow data for one or more flows that correspond to the device-circuit pair;

calculate a time difference for each flow of the one or more flows that correspond to the device-circuit pair;

based on the calculated time differences and the received flow data, update a probability distribution model associated with the device-circuit pair; and

determine whether a time bucket, of the time series, is complete or open based on the updated probability distribution model.

11 . The system of claim 10 , wherein the at least one processor is further configured to:

in response to determining that the time bucket is determined to be complete, ignore further flow data that corresponds to the time bucket; and

in response to determining that the time bucket is determined to be open, incorporate further flow data that corresponds to the time bucket.

12 . The system of claim 11 , wherein the at least one processor is further configured to:

in response to determining that the time bucket is determined to be complete, send flow data that corresponds to the time bucket to a detection module to detect possible network anomalies.

13 . The system of claim 10 , wherein the probability distribution model comprises flow data that corresponds to the device-circuit pair and time differences for flows that correspond to the device-circuit pair.

14 . The system of claim 13 , wherein the at least one processor is further configured to update the probability distribution model by:

incorporating the received flow data and the calculated time differences into the probability distribution model;

calculating a mean value based on the time differences and the flow data included in the probability distribution model; and

calculating a standard deviation value based the time differences and the flow data included in the probability distribution model.

15 . The system of claim 14 , wherein the at least one processor is further configured to determine whether the time bucket is complete or open by:

calculating a time delay value based on the standard deviation value; and

determining whether the time bucket is complete or open based on the time delay value and a file stamp time value of a network flow record containing the received flow data.

16 . The system of claim 15 , wherein the at least one processor is further configured to calculate the time delay value by calculating the time delay value based on the standard deviation value and the mean value.

17 . The system of claim 15 , wherein the at least one processor is further configured to determine whether the time bucket is complete or open by:

creating an expiry time based on an end time of the time bucket and the calculated time delay value;

determining that the time bucket is complete if the file stamp time is beyond the created expiry time; and

determining that the time bucket is open if the file stamp time is not beyond the created expiry time.

18 . The system of claim 13 , wherein each of the time differences in the probability distribution model is a time difference between a start time of each flow in the probability distribution model and a file stamp time of a corresponding network flow record.

19 . A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations for processing network flow data over a time series associated with a device-circuit pair, comprising:

receiving flow data for one or more flows that correspond to the device-circuit pair;

calculating a time difference for each flow of the one or more flows that correspond to the device-circuit pair;

based on the calculated time differences and the received flow data, updating a probability distribution model associated with the device-circuit pair; and

determining whether a time bucket, of the time series, is complete or open based on the updated probability distribution model.

20 . The non-transitory computer-readable medium of claim 19 , further comprising:

in response to determining that the time bucket is determined to be complete, ignoring further flow data that corresponds to the time bucket; and

in response to determining that the time bucket is determined to be open, incorporating further flow data that corresponds to the time bucket.