Recursive agglomerative clustering of time-structured communications
An example method of document cluster labeling comprises: selecting a current document cluster of a plurality of document clusters (e.g., the current document cluster can have documents organized using a DBSCAN or OPTICS algorithm); initializing a label associated with the current document cluster; selecting a term from a list of terms comprised by the document cluster; appending the term to the label associated with the current document cluster; responsive to determining that the label is found in a label dictionary, iteratively selecting a next term from the list of terms comprised by the document cluster and appending the next term to the label associated with the current document cluster; responsive to failing to locate the label in the label dictionary, inserting the label into the label dictionary; and associating the label with the current document cluster.
1 . A method of document cluster labeling, the method comprising:
selecting, by a processing device, a current document cluster of a plurality of document clusters, wherein the current document cluster includes documents organized according to a predefined algorithm based on respective distances between vectorized representations of the respective documents within the current document cluster;
based on a vector representing the current document cluster, identifying a sorted list of terms associated with the current document cluster, wherein respective terms of the sorted list of terms are weighted according to a Term Frequency-Inverse Document Frequency (TF-IDF) weighting scheme;
initializing a label associated with the current document cluster;
automatically selecting a first term from the list of terms comprised by the documents of the document cluster, based on respective weightings of the terms according to the term frequency-inverse document frequency (TF-IDF) weighting scheme;
automatically appending the term to the label associated with the current document cluster;
responsive to determining that the label is found in a label dictionary, (i) iteratively selecting one or more next terms from the list of terms comprised by the document cluster based on the respective weightings of the terms according to the TF-IDF weighting scheme and (ii) automatically appending each of the next terms to the label associated with the current document cluster;
responsive to failing to locate the label in the label dictionary, inserting the label into the label dictionary; and
associating the label with the current document cluster.
2 . The method of claim 1 , further comprising sorting the plurality of document clusters by a number of documents comprised by a respective document cluster.
3 . The method of claim 1 , further comprising sorting the list of terms by a respective term weight.
4 . The method of claim 1 , further comprising excluding, from the list of terms, a term comprising at least part of an entity name.
5 . The method of claim 1 , further comprising visually representing, via a graphical user interface, one or more clusters of the plurality of document clusters in a visual association with respective labels.
6 . The method of claim 1 , wherein the plurality of document clusters comprises a plurality of electronic mail messages.
7 . The method of claim 1 , wherein the plurality of document clusters comprises a plurality of documents represented by respective vectors in a hyperspace of document features.
8 . A system, comprising:
a memory; and
a processor coupled to the memory, wherein the processor is configured to:
select a current document cluster of a plurality of document clusters, wherein the current document cluster includes documents organized according to a predefined algorithm based on respective distances between vectorized representations of the respective documents within the current document cluster;
based on a vector representing the current document cluster, identify a sorted list of terms associated with the current document cluster, wherein respective terms of the sorted list of terms are weighted according to a Term Frequency-Inverse Document Frequency (TF-IDF) weighting scheme;
initialize a label associated with the current document cluster;
automatically select a first term from the list of terms comprised by the documents of the document cluster, based on respective weightings of the terms according to the term frequency-inverse document frequency (TF-IDF) weighting scheme;
automatically append the term to the label associated with the current document cluster;
responsive to determining that the label is found in a label dictionary, (i) iteratively select one or more next terms from the list of terms comprised by the document cluster based on the respective weightings of the terms according to the TF-IDF weighting scheme and (ii) automatically append each of the next terms to the label associated with the current document cluster;
responsive to failing to locate the label in the label dictionary, insert the label into the label dictionary; and
associate the label with the current document cluster.
9 . The system of claim 8 , wherein the processor is further configured to sort the plurality of document clusters by a number of documents comprised by a respective document cluster.
10 . The system of claim 8 , wherein the processor is further configured to sort the list of terms by a respective term weight.
11 . The system of claim 8 , wherein the processor is further configured to exclude, from the list of terms, a term comprising at least part of an entity name.
12 . The system of claim 8 , wherein the processor is further configured to visually represent, via a graphical user interface, one or more clusters of the plurality of document clusters in a visual association with respective labels.
13 . The system of claim 8 , wherein the plurality of document clusters comprises a plurality of electronic mail messages.
14 . The system of claim 8 , wherein the plurality of document clusters comprises a plurality of documents represented by respective vectors in a hyperspace of document features.
15 . A non-transitory, computer-readable storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to:
select a current document cluster of a plurality of document clusters, wherein the current document cluster includes documents organized according to a predefined algorithm based on respective distances between vectorized representations of the respective documents within the current document cluster;
based on a vector representing the current document cluster, identify a sorted list of terms associated with the current document cluster, wherein respective terms of the sorted list of terms are weighted according to a Term Frequency-Inverse Document Frequency (TF-IDF) weighting scheme;
initialize a label associated with the current document cluster;
automatically select a first term from the list of terms comprised by the documents of the document cluster, based on respective weightings of the terms according to the term frequency-inverse document frequency (TF-IDF) weighting scheme;
automatically append the term to the label associated with the current document cluster;
responsive to determining that the label is found in a label dictionary, (i) iteratively select one or more next terms from the list of terms comprised by the document cluster based on the respective weightings of the terms according to the TF-IDF weighting scheme and (ii) automatically append each of the next terms to the label associated with the current document cluster;
responsive to failing to locate the label in the label dictionary, insert the label into the label dictionary; and
associate the label with the current document cluster.
16 . The non-transitory, computer-readable storage medium of claim 15 , further comprising executable instructions that, when executed by the computer system, cause the computer system to sort the plurality of document clusters by a number of documents comprised by a respective document cluster.
17 . The non-transitory, computer-readable storage medium of claim 15 , further comprising executable instructions that, when executed by the computer system, cause the computer system to sort the list of terms by a respective term weight.
18 . The non-transitory, computer-readable storage medium of claim 15 , further comprising executable instructions that, when executed by the computer system, cause the computer system to exclude, from the list of terms, a term comprising at least part of an entity name.
19 . The non-transitory, computer-readable storage medium of claim 15 , further comprising executable instructions that, when executed by the computer system, cause the computer system to visually represent, via a graphical user interface, one or more clusters of the plurality of document clusters in a visual association with respective labels.
20 . The non-transitory, computer-readable storage medium of claim 15 , wherein the plurality of document clusters comprises a plurality of electronic mail messages.
21 . The method of claim 1 , wherein:
the current document cluster is grouped based on a generating vectors for each respective document of the current document cluster including (i) a time-sensitive factor and (ii) a content-sensitive factor, and
the weighting of the terms according to the TF-IDF weighting scheme is based on the content-sensitive factor.
22 . The method of claim 1 , wherein:
the predefined algorithm comprises a DBSCAN algorithm such that (i) each document within the current document cluster satisfies a nearby-neighbor threshold and (ii) each document within the current document cluster is connected to each other document within the current document cluster via a path that is contained completely within the current document cluster and that passes through a core of the current document cluster; or
the predefined algorithm comprises an OPTICS algorithm such that each document within the current document cluster is linearly ordered with respect to each other document within the current document cluster based on a spatial distance between each of the documents within the current document cluster.