Graph based framework for detecting malicious or compromised accounts
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting malicious attacks. One of the methods includes generating a collection of hypergraphs representing user events across a collection of users; analyzing the collection of hypergraphs to determine a group of malicious user accounts or account activities satisfying a threshold confidence; using the group of malicious user accounts or account activities as training data for a machine learning system that generates one or more classifiers; and using the one or more generated classifiers to output additional malicious user accounts or account activities.
1. A method comprising:
generating a collection of hypergraphs representing user events across a collection of users, wherein each hypergraph node corresponds to a feature profile computed from a set of correlated events or users and wherein each edge between hypergraph nodes corresponds to attributes specifying a relationship between the hypergraph nodes, wherein generating the collection of hypergraphs includes obtaining event log data associated with the collection of users including one or more of login logs, signup logs, or transaction logs;
analyzing the collection of hypergraphs to determine an initial group of malicious user accounts or account activities satisfying a threshold confidence;
using the initial group of malicious user accounts or account activities as first training data for a machine learning system and a group of user accounts or account activities not identified as malicious as second training data for the machine learning system, wherein the training generates one or more classifiers configured to classify user accounts or account activities as malicious based on feature vectors derived from the first and second training data; and
using the one or more generated classifiers on a collection of unclassified user accounts and account activities to output additional malicious user accounts or account activities in addition to those identified in the analysis of the collection of hypergraphs.
2. The method of claim 1 , wherein each hypergraph includes nodes corresponding to a feature profile associated with user accounts or events and edges indicating a relationship between nodes.
3. The method of claim 1 , wherein analyzing the collection of hypergraphs to determine the initial group of malicious user accounts or account activities comprises:
applying one or more community detection techniques to the hyper graphs to identify suspicious sub-graph components;
determining that the nodes associated with the suspicious sub-graph components are suspicious; and
outputting accounts or events associated with the suspicious sub-graph components as candidate malicious accounts or events.
4. The method of claim 3 , further comprising examining the candidate accounts or events using a set of one or more rules or a whitelist to filter potential false positive accounts or events.
5. The method of claim 1 , wherein analyzing the collection of hypergraphs to determine the initial group of malicious user accounts or account activities comprises:
assigning a suspiciousness score to each node of the hypergraphs, wherein each node corresponds to a feature profile associated with user accounts or events;
applying one or more graph diffusion techniques to the hyper graphs; and
selecting a set of one or more nodes with high suspiciousness scores as candidate malicious accounts or events.
6. The method of claim 1 , wherein using the group of malicious user accounts or account activities as training data for a machine learning system that generates one or more classifiers further includes obtaining additional user accounts or account activities to use as good training data.
7. The method of claim 6 , wherein the good training data and group of malicious user accounts or account activities are used to derive a set of rich features used to generate the one or more classifiers.
8. The method of claim 1 , further comprising using the output additional malicious user accounts or account activities to derive a set of signals to combine with the one or more classifiers to provide real-time detection of future user events or user accounts.
9. A system comprising:
one or more computers configured to perform operations comprising:
generating a collection of hypergraphs representing user events across a collection of users, wherein each hypergraph node corresponds to a feature profile computed from a set of correlated events or users and wherein each edge between hypergraph nodes corresponds to attributes specifying a relationship between the hypergraph nodes, wherein generating the collection of hypergraphs includes obtaining event log data associated with the collection of users including one or more of login logs, signup logs, or transaction logs;
analyzing the collection of hypergraphs to determine an initial group of malicious user accounts or account activities satisfying a threshold confidence;
using the initial group of malicious user accounts or account activities as training data for a machine learning system and a group of user accounts or account activities not identified as malicious as second training data for the machine learning system, wherein the training generates one or more classifiers configured to classify user accounts or account activities as malicious based on feature vectors derived from the first and second training data; and
using the one or more generated classifiers on a collection of unclassified user accounts and account activities to output additional malicious user accounts or account activities in addition to those identified in the analysis of the collection of hypergraphs.
10. The system of claim 9 , wherein each hypergraph includes nodes corresponding to a feature profile associated with user accounts or events and edges indicating a relationship between nodes.
11. The system of claim 9 , wherein analyzing the collection of hypergraphs to determine a group of malicious user accounts or account activities comprises:
applying one or more community detection techniques to the hyper graphs to identify suspicious sub-graph components;
determining that the nodes associated with the suspicious sub-graph components are suspicious; and
outputting accounts or events associated with the suspicious sub-graph components as candidate malicious accounts or events.
12. The system of claim 11 , further comprising examining the candidate accounts or events using a set of one or more rules or a whitelist to filter potential false positive accounts or events.
13. The system of claim 9 , wherein analyzing the collection of hypergraphs to determine a group of malicious user accounts or account activities comprises:
assigning a suspiciousness score to each node of the hypergraphs, wherein each node corresponds to a feature profile associated with user accounts or events;
applying one or more graph diffusion techniques to the hyper graphs; and
selecting a set of one or more nodes with high suspiciousness scores as candidate malicious accounts or events.
14. The system of claim 9 , using the group of malicious user accounts or account activities as training data for a machine learning system that generates one or more classifiers further includes obtaining additional user accounts or account activities to use as good training data.
15. The system of claim 14 , wherein the good training data and group of malicious user accounts or account activities are used to derive a set of rich features used to generate the one or more classifiers.
16. The system of claim 9 , further configured to perform operations comprising using the output additional malicious user accounts or account activities to derive a set of signals to combine with the one or more classifiers to provide real-time detection of future user events or user accounts.