Anomaly detection in streaming graphs
Techniques for detecting anomalies in streaming graph data are described. For example, an embedding technique of generating a multi-dimensional vector of summations of each weighted edge found in both a random source bounding proper subset and a random destination bounding proper subset associated with a dimension of the epoch graph is detailed. Anomaly detection is performed on the generated multi-dimensional vectors.
1. A computer-implemented method comprising:
collecting streaming graph data into a plurality of epoch graphs;
for each epoch graph, performing an embedding to generate a multi-dimensional vector, wherein each element of the multi-dimensional vector is a summation of each weighted edge found in both a random source bounding subset and a random destination bounding subset associated with a dimension of the epoch graph;
performing anomaly detection on the generated multi-dimensional vectors; and
generating an output indicating any anomaly detected.
2. The computer-implemented method of claim 1 , wherein each random source bounding subset is selected by an application of a unique hash function to nodes of the epoch graph and the random destination bounding subset is selected by an application of a unique hash function to nodes of the epoch graph.
3. The computer-implemented method of claim 1 , further comprising:
sequentially evaluating, per dimension, each weighted edge of the epoch graph to determine which weighted edges are found in both the random source bounding subset and the random destination bounding subset.
4. A computer-implemented method comprising:
for each epoch graph of streaming graph data, performing an embedding to generate a multi-dimensional vector, wherein each element of the multi-dimensional vector is a summation of each weighted edge found in both a random source bounding proper subset and a random destination bounding proper subset associated with a dimension of the epoch graph; and
performing anomaly detection on the generated multi-dimensional vectors.
5. The computer-implemented method of claim 4 , wherein each random source bounding proper subset is determined by an application of a unique hash function to nodes of the epoch graph and the random destination bounding proper subset is determined by an application of a unique hash function to nodes of the epoch graph.
6. The computer-implemented method of claim 4 , further comprising:
sequentially evaluating, per dimension, all weighted edges of the epoch graph to determine which weighted edges are found in both the random source bounding proper subset and the random destination bounding proper subset.
7. The computer-implemented method of claim 4 , further comprising:
sequentially evaluating, per dimension, each vertex of the epoch graph to determine which weighted edges are found in both the random source bounding proper subset and the random destination bounding proper subset associated.
8. The computer-implemented method of claim 4 , further comprising:
generating an output indicating any anomaly detected.
9. The computer-implemented method of claim 8 , wherein the output is at least one of a graph or an alarm.
10. The computer-implemented method of claim 4 , wherein the anomaly detection is a random cut forest algorithm.
11. The computer-implemented method of claim 4 , wherein each weighted edge comprises data regarding at least one of: port information, packet information, protocol, or timestamp.
12. The computer-implemented method of claim 4 , further comprising:
filtering each epoch graph of streaming graph data to remove extraneous information prior to performing an embedding.
13. A system comprising:
storage for a plurality of epoch graphs; and
an anomaly detection service implemented by a first one or more electronic devices, the anomaly detection service to, for each epoch graph,
perform an embedding to generate a multi-dimensional vector, wherein each element of the multi-dimensional vector is a summation of each weighted edge found in both a random source bounding proper subset and a random destination bounding proper subset associated with a dimension of the epoch graph, and
perform anomaly detection on the generated multi-dimensional vectors.
14. The system of claim 13 , wherein each random source bounding proper subset is determined by an application of a unique hash function to nodes of the epoch graph and the random destination bounding proper subset is determined by an application of a unique hash function to nodes of the epoch graph.
15. The system of claim 13 , wherein the anomaly detection service is further to sequentially evaluate, per dimension, all weighted edges of the epoch graph to determine which weighted edges are found in both the random source bounding proper subset and the random destination bounding proper subset.
16. The system of claim 13 , wherein the anomaly detection service is further to sequentially evaluate, per dimension, each vertex of the epoch graph to determine which weighted edges are found in both the random source bounding proper subset and the random destination bounding proper subset associated.
17. The system of claim 13 , wherein the anomaly detection service is further to generate an output indicating any anomaly detected.
18. The system of claim 17 , wherein the output is at least one of a graph or an alarm.
19. The system of claim 13 , wherein the anomaly detection is a random cut forest algorithm.
20. The system of claim 13 , wherein each weighted edge comprises data regarding at least one of: port information, packet information, protocol, or timestamp.