Graph database analysis for network anomaly detection systems
Graph database analysis for network anomaly detection systems, in which a data analysis device receives multiple log data entries including parameters associated with a computer network event in a computing network. The data analysis device extracts one or more parameters in real-time and generates a network event graph based on at least one of a first graph metric or a second graph metric. The first and second graph metrics are based on the one or more extracted parameters. The data analysis device detects, based on queries performed on the network event graph, at least one of an anomalous event associated with the computing network or a malicious event associated with the computing network.
1. A method for detecting malicious computer network events using graph database analysis, the method comprising:
receiving log data that includes parameters describing activity in a network;
generating, based on the parameters, an event graph comprising information about network activity involving a primary computing node in the network;
storing the event graph in a graph database of the network in response to generating the event graph based on the parameters;
processing the event graph using a graphics processing unit (“GPU”) that is coupled to the graph database to accelerate graphical analysis of information representing event graphs stored in the graph database;
detecting an occurrence of a malicious event that involves the primary computing node at the network in response to processing the information about the network activity using the GPU coupled to the graph database;
after detecting the occurrence of the malicious event that involves the primary computing node, selecting, from among all nodes in the network, a secondary computing node that has exchanged data packets directly with the primary computing node at the network in which the occurrence of the malicious event has been detected;
selecting, from among all of the nodes of the network other than the primary computing node at the network in which the occurrence of the malicious event has been detected, a tertiary computing node that has exchanged data packets directly with the secondary computing node and that has not exchanged data packets directly with the primary computing node;
assigning a first risk score to the secondary computing node based on the secondary computing having exchanged data packets directly with the primary computing node, and a reduced, second risk score to the tertiary computing node based on the tertiary computing node having exchanged data packets directly with the secondary computing node and the tertiary computing node having not exchanged data packets directly with the primary computing node.
2. The method of claim 1 , wherein the network activity involves multiple distinct nodes in the network and processing the event graph comprises:
determining a risk level of the network activity involving the multiple distinct nodes in response to processing the information about the network activity using the GPU coupled to the graph database.
3. The method of claim 2 , wherein determining the risk level of the network activity comprises:
generating a respective risk score for each node of the multiple distinct nodes in response to accelerating graphical analysis of the information about the network activity using the GPU coupled to the graph database; and
determining the risk level of the network activity using the respective risk scores for each node of the multiple distinct nodes.
4. The method of claim 3 , comprising:
generating a node structure based on the respective risk scores for each node of the multiple distinct nodes; and
based on the node structure, identifying one or more types of ordered-connections that exist between two or more nodes of the multiple distinct nodes.
5. The method of claim 4 , wherein detecting the occurrence of the malicious event comprises:
detecting the occurrence of the malicious event based on:
a particular type of ordered-connection that exists between two distinct nodes of the multiple distinct nodes; and
the respective risk score for each of the two nodes.
6. The method of claim 1 , comprising:
structuring the log data in a graphical format in response to determining that the parameters in the log data have a data format that conflicts with a graphical data format of the graph database; and
providing a data stream of parameters to the graph database in response to extracting the parameters from the log data as part of structuring the log data.
7. The method of claim 6 , wherein storing the event graph in the graph database comprises:
storing the event graph as a graphical data structure configured for accelerated processing by the GPU coupled to the graph database, in response to structuring the log data in the graphical format.
8. The method of claim 6 , wherein structuring the log data comprises:
using a stream analytics engine to apply an extraction function to a subset of the log data to extract the parameters from the subset of the log data; and
processing, by an intermediate storage device, the parameters extracted from the subset of the log data for conversion to a graphical file format that is accepted by the graph database in response to receiving the extracted parameters from the stream analytics engine.
9. The method of claim 1 , wherein processing the event graph comprises:
processing, at the graph database, a plurality of event graphs stored in the graph database by using the GPU to accelerate graphical analysis of information representing each event graph of the plurality of event graphs.
10. A system for detecting malicious computer network events using graph database analysis, the system comprising:
one or more processing devices;
one or more non-transitory machine-readable storage devices storing instructions that are executable by the one or more processing devices to cause performance of operations comprising:
receiving log data that includes parameters describing activity in a network;
generating, based on the parameters, an event graph comprising information about network activity involving a primary computing node in the network;
storing the event graph in a graph database of the network in response to generating the event graph based on the parameters;
processing the event graph using a graphics processing unit (“GPU”) that is coupled to the graph database to accelerate graphical analysis of information representing event graphs stored in the graph database;
detecting an occurrence of a malicious event that involves the primary computing node at the network in response to processing the information about the network activity using the GPU coupled to the graph database;
after detecting the occurrence of the malicious event that involves the primary computing node, selecting, from among all nodes in the network, a secondary computing node that has exchanged data packets directly with the primary computing node at the network in which the occurrence of the malicious event has been detected;
selecting, from among all of the nodes of the network other than the primary computing node at the network in which the occurrence of the malicious event has been detected, a tertiary computing node that has exchanged data packets directly with the secondary computing node and that has not exchanged data packets directly with the primary computing node;
assigning a first risk score to the secondary computing node based on the secondary computing having exchanged data packets directly with the primary computing node, and a reduced, second risk score to the tertiary computing node based on the tertiary computing node having exchanged data packets directly with the secondary computing node and the tertiary computing node having not exchanged data packets directly with the primary computing node.
11. The system of claim 10 , wherein the network activity involves multiple distinct nodes in the network and processing the event graph comprises:
determining a risk level of the network activity involving the multiple distinct nodes in response to processing the information about the network activity using the GPU coupled to the graph database.
12. The system of claim 11 , wherein determining the risk level of the network activity comprises:
generating a respective risk score for each node of the multiple distinct nodes in response to accelerating graphical analysis of the information about the network activity using the GPU coupled to the graph database; and
determining the risk level of the network activity using the respective risk scores for each node of the multiple distinct nodes.
13. The system of claim 12 , the operations comprise:
generating a node structure based on the respective risk scores for each node of the multiple distinct nodes; and
based on the node structure, identifying one or more types of ordered-connections that exist between two or more nodes of the multiple distinct nodes.
14. The system of claim 13 , wherein detecting the occurrence of the malicious event comprises:
detecting the occurrence of the malicious event based on:
a particular type of ordered-connection that exists between two distinct nodes of the multiple distinct nodes; and
the respective risk score for each of the two nodes.
15. The system of claim 10 , the operations comprise:
structuring the log data in a graphical format in response to determining that the parameters in the log data have a data format that conflicts with a graphical data format of the graph database; and
providing a data stream of parameters to the graph database in response to extracting the parameters from the log data as part of structuring the log data.
16. The system of claim 15 , wherein storing the event graph in the graph database comprises:
storing the event graph as a graphical data structure configured for accelerated processing by the GPU coupled to the graph database, in response to structuring the log data in the graphical format.
17. The system of claim 15 , wherein structuring the log data comprises:
using a stream analytics engine to apply an extraction function to a subset of the log data to extract the parameters from the subset; and
processing, by an intermediate storage device, the parameters extracted from the subset for conversion to a graphical file format that is accepted by the graph database in response to receiving the extracted parameters from the stream analytics engine.
18. The system of claim 10 , wherein processing the event graph comprises:
processing, at the graph database, a plurality of event graphs stored in the graph database by using the GPU to accelerate graphical analysis of information representing each event graph of the plurality of event graphs.
19. A non-transitory machine-readable storage device storing instructions that are executable by a processing device to cause performance of operations comprising:
receiving log data that includes parameters describing activity in a network;
generating, based on the parameters, an event graph comprising information about network activity involving a primary computing node in the network;
storing the event graph in a graph database of the network in response to generating the event graph based on the parameters;
processing the event graph using a graphics processing unit (“GPU”) that is coupled to the graph database to accelerate graphical analysis of information representing event graphs stored in the graph database;
detecting an occurrence of a malicious event that involves the primary computing node at the network in response to processing the information about the network activity using the GPU coupled to the graph database;
after detecting the occurrence of the malicious event that involves the primary computing node, selecting, from among all nodes in the network, a secondary computing node that has exchanged data packets directly with the primary computing node at the network in which the occurrence of the malicious event has been detected;
selecting, from among all of the nodes of the network other than the primary computing node at the network in which the occurrence of the malicious event has been detected, a tertiary computing node that has exchanged data packets directly with the secondary computing node and that has not exchanged data packets directly with the primary computing node;
assigning a first risk score to the secondary computing node based on the secondary computing having exchanged data packets directly with the primary computing node, and a reduced, second risk score to the tertiary computing node based on the tertiary computing node having exchanged data packets directly with the secondary computing node and the tertiary computing node having not exchanged data packets directly with the primary computing node.
20. The machine-readable storage device of claim 19 , wherein the network activity involves multiple distinct nodes in the network and processing the event graph comprises:
determining a risk level of the network activity involving the multiple distinct nodes in response to processing the information about the network activity using the GPU coupled to the graph database.