Systems and methods for clustering data to improve data analytics
A computer-implemented method for clustering data to improve data analytics may include (1) extracting a social graph from a data set of messages, the social graph indicating messages as edges such that nodes of the edges indicate corresponding senders and recipients in sender-recipient relationships, (2) detecting communities of collaborators by identifying clusters of nodes within the social graph, (3) applying the identified clusters of nodes within the social graph to a grouping calculation to group the messages of the data set into groups of messages, and (4) providing, through a computing interface, results of a data analytics operation to an end user based at least in part on applying the identified clusters of nodes within the social graph to the grouping calculation. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for clustering data to improve data analytics, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
extracting a social graph from a data set of messages, the social graph indicating messages as edges within the social graph such that nodes of the edges indicate corresponding senders and recipients in sender-recipient relationships;
detecting communities of collaborators by identifying clusters of nodes within the social graph by minimizing a number of edges that cross between the clusters by performing a smart user replication operation to replicate a user between clusters such that there are no edges that cross between different clusters;
applying the identified clusters of nodes within the social graph to a grouping calculation to group the messages of the data set into groups of messages at least in part by creating, for each respective one of the identified clusters of nodes within the social graph, a respective group of messages by adding each message that identifies two separate users from the respective one of the identified clusters in at least one address field of the message to the respective group of messages;
providing, through a computing interface, results of a data analytics operation to an end user based at least in part on applying the identified clusters of nodes within the social graph to the grouping calculation to group the messages of the data set into the groups of messages;
wherein:
the method further comprises decoupling a user plane from a data plane;
the user plane includes meta-data indicating sender-recipient relationships; and
the data plane includes actual underlying content of the messages.
2. The computer-implemented method of claim 1 , wherein performing the method improves an efficiency of data clustering according to the Davies-Bouldin index in comparison to another clustering system.
3. The computer-implemented method of claim 2 , wherein performing the method improves the efficiency of data clustering according to the Davies-Bouldin index in comparison to CLUTO.
4. The computer-implemented method of claim 1 , wherein creating each respective group of messages is performed based on an analysis of just address field values for the messages rather than an analysis of bodies of the messages.
5. The computer-implemented method of claim 1 , wherein the method operates as a single pass algorithm rather than an iterative clustering algorithm.
6. The computer-implemented method of claim 1 , wherein the data analytics operation identifies the top N closest collaborators to a specified user where N is an arbitrary natural number.
7. The method of claim 1 , wherein the data analytics operation comprises an e-discovery operation.
8. The method of claim 1 , the data analytics operation comprises a digital forensics operation.
9. The method of claim 8 , wherein the data analytics operation further comprises an e-discovery operation.
10. The method of claim 1 , wherein the data analytics operation comprises a data retention operation.
11. A system for clustering data to improve data analytics, the system comprising:
an extraction module, stored in memory, that extracts a social graph from a data set of messages, the social graph indicating messages as edges within the social graph such that nodes of the edges indicate corresponding senders and recipients in sender-recipient relationships;
a detection module, stored in memory, that detects communities of collaborators by identifying clusters of nodes within the social graph by minimizing a number of edges that cross between the clusters by performing a smart user replication operation to replicate a user between clusters such that there are no edges that cross between different clusters;
an application module, stored in memory, that applies the identified clusters of nodes within the social graph to a grouping calculation to group the messages of the data set into groups of messages at least in part by creating, for each respective one of the identified clusters of nodes within the social graph, a respective group of messages by adding each message that identifies two separate users from the respective one of the identified clusters in at least one address field of the message to the respective group of messages;
a provisioning module, stored in memory, that provides, through a computing interface, results of a data analytics operation to an end user based at least in part on applying the identified clusters of nodes within the social graph to the grouping calculation to group the messages of the data set into the groups of messages;
at least one physical processor configured to execute the extraction module, the detection module, the application module, and the provisioning module;
wherein:
the extraction module further decouples a user plane from a data plane;
the user plane includes meta-data indicating sender-recipient relationships; and
the data plane includes actual underlying content of the messages.
12. The system of claim 11 , wherein the messages comprise at least one of:
emails;
email attachments.
13. The system of claim 11 , wherein the data analytics operation comprises at least one of:
an e-discovery operation;
a digital forensics operation;
a data retention operation.
14. The system of claim 13 , wherein the data analytics operation comprises the e-discovery operation.
15. The system of claim 13 , wherein the data analytics operation comprises the digital forensics operation.
16. The system of claim 13 , wherein the data analytics operation comprises the data retention operation.
17. The system of claim 11 , wherein the extraction module extracts the social graph based on a value in at least one of the following fields:
a sender address field in a message in the messages;
a recipient address field in the message in the messages;
a carbon copy address field in the message in the messages;
a blind carbon copy address field in the message in the messages.
18. The system of claim 11 , wherein the identified clusters of nodes within the social graph indicate separate departments within an enterprise organization such that members of a department tend to message other members within the same department rather than a different department according to a statistical measurement.
19. The system of claim 18 , wherein the data analytics operation discovers departments within the enterprise organization that were previously unknown to the system performing the data analytics operation.
20. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
extract a social graph from a data set of messages, the social graph indicating messages as edges within the social graph such that nodes of the edges indicate corresponding senders and recipients in sender-recipient relationships;
detect communities of collaborators by identifying clusters of nodes within the social graph by minimizing a number of edges that cross between the clusters by performing a smart user replication operation to replicate a user between clusters such that there are no edges that cross between different clusters;
apply the identified clusters of nodes within the social graph to a grouping calculation to group the messages of the data set into groups of messages at least in part by creating, for each respective one of the identified clusters of nodes within the social graph, a respective group of messages, by adding each message that identifies two separate users from the respective one of the identified clusters in at least one address field of the message to the respective group of messages;
provide, through a computing interface, results of a data analytics operation to an end user based at least in part on applying the identified clusters of nodes within the social graph to the grouping calculation to group the messages of the data set into the groups of messages;
wherein:
the instructions cause the computing device to decouple a user plane from a data plane;
the user plane includes meta-data indicating sender-recipient relationships; and
the data plane includes actual underlying content of the messages.