Message classification
One or more computing devices, systems, and/or methods for message classification are provided. For example, a set of messages is clustered into a set of clusters. A cluster comprises messages with similar features (e.g., similar subject lines, message body content, sender information, recipient information, structure, user action such as reading or deleting, spam vote information, etc.). Cluster features are computed for the clusters based upon features of messages within such clusters. A first table, comprising cluster entries corresponding cluster features of clusters, and a second table, comprising message entries corresponding to clusters to which messages are assigned, are created. Message features of a message are created, using the first table and second table, based upon features of the message and cluster features of clusters to which the message is assigned. A message classifier is used to classify the message (e.g., spam, safe, a threat, etc.) based upon the message features.
1. A method of message classification, the method comprising:
executing, on a processor of a computing device, instructions that cause the computing device to perform operations, the operations comprising:
clustering messages into clusters based upon hash descriptions of bags of words corresponding to message components of the messages;
computing cluster features for the clusters based upon one or more features of one or more messages within the clusters;
creating a first table comprising cluster entries for the clusters, wherein a first cluster entry of a first cluster is populated with one or more cluster features derived from one or more message components of at least one message clustered within the first cluster;
creating a second table populated with message entries for the messages, wherein a first message entry, of the message entries in the second table, for a first message is populated with identifiers of one or more clusters into which the first message is clustered;
querying the first table and the second table to create one or more message features for a second message based upon one or more second features of the second message and one or more second cluster features of one or more second clusters to which the second message is assigned;
training a message classifier to use a learned decision rule that is trained using a set of training message data labeled based upon one or more spam indicators;
balancing a number of positive spam training message examples with a number of negative non-spam training message examples, in association with the message classifier, by subsampling a ratio of negative non-spam training message examples; and
classifying the second message based upon the one or more message features and the learned decision rule of the message classifier.
2. The method of claim 1 , wherein the computing cluster features comprises:
measuring a distance between a first bag of words and a second bag of words within a cluster to create a cluster feature.
3. The method of claim 1 , wherein the computing cluster features comprises:
measuring a number of messages within a cluster to create a cluster feature.
4. The method of claim 1 , wherein the computing cluster features comprises:
aggregating spam scores for messages within a cluster, recipient characteristic features of recipients of the messages within the cluster, user action features performed upon the messages within the cluster by the recipients, message content features of the messages within the cluster, and subject line features of the messages within the cluster to create the cluster features.
5. The method of claim 1 , wherein the clustering comprises:
clustering the messages within a sender description space, a subject line space, a message body content space, a user action space, and an extensible markup language (XML) document object model (DOM) structure space for message body content.
6. The method of claim 1 , wherein the clustering comprises:
clustering the messages based upon a user read action feature, a user reply action feature, a user forward action feature, a user delete action feature, and a user spam vote feature.
7. The method of claim 1 , comprising:
transforming, using a hash function, a bag of words into a min-hash description based upon a number of words having a minimum hash value.
8. The method of claim 1 , comprising:
retroactively classifying the second message after the second message was delivered to a message inbox.
9. The method of claim 8 , comprising:
responsive to classifying the second message as spam, moving the second message from the message inbox to a spam folder.
10. The method of claim 1 , wherein the clustering comprises:
utilizing a locality sensitive hashing technique for clustering the messages.
11. The method of claim 1 , wherein the clustering comprises:
clustering the first message into both the first cluster and into a second cluster.
12. The method of claim 1 , comprising:
if a training message receives a spam vote spam indicator within a threshold timespan, then designating the training message as spam, otherwise, designating the training message as not spam.
13. The method of claim 11 ,
wherein identifiers of the first cluster and the second cluster are populated within the first message entry.
14. The method of claim 1 , comprising:
using the message classifier to identify a spam campaign.
15. A computing device comprising:
a processor; and
memory comprising processor-executable instructions that when executed by the processor cause performance of operations, the operations comprising:
clustering messages into clusters based upon features of the messages;
computing cluster features for the clusters based upon one or more features of one or more messages within the clusters;
creating a first table comprising cluster entries for the clusters, wherein a first cluster entry of a first cluster is populated with one or more cluster features derived from one or more message components of at least one message clustered within the first cluster;
creating a second table populated with message entries for the messages, wherein a first message entry, of the message entries in the second table, for a first message is populated with identifiers of one or more clusters into which the first message is clustered;
querying the first table and the second table to create one or more message features for a second message based upon one or more second features of the second message and one or more second cluster features of one or more second clusters to which the second message is assigned;
training a message classifier to use a learned decision rule that is trained using a set of training message data labeled based upon one or more spam indicators;
balancing a number of positive spam training message examples with a number of negative non-spam training message examples, in association with the message classifier, by subsampling a ratio of negative non-spam training message examples; and
classifying the second message based upon the one or more message features and the learned decision rule of the message classifier.
16. The computing device of claim 15 , wherein the one or more message features of the second message are created based upon a number of messages within a cluster to which the second message is assigned, user actions upon messages within the cluster, subject line features of the messages within the cluster, sender description features of senders of the messages within the cluster, word content of message bodies of the messages within the cluster, message body structures of the messages within the cluster, and spam filtering scores for the messages within the cluster.
17. The computing device of claim 15 , wherein the operations comprise:
generating a message feature based upon an aggregate of features of two or more clusters to which the second message is assigned.
18. A non-transitory machine readable medium having stored thereon processor-executable instructions that when executed cause performance of operations, the operations comprising:
clustering messages into clusters;
computing cluster features for the clusters based upon one or more features of one or more messages within the clusters;
creating a first table comprising cluster entries for the clusters, wherein a first cluster entry of a first cluster is populated with one or more cluster features derived from one or more message components of at least one message clustered within the first cluster;
creating a second table populated with message entries for the messages, wherein a first message entry, of the message entries in the second table, for a first message is populated with identifiers of one or more clusters into which the first message is clustered;
querying the first table and the second table to create one or more message features for a second message based upon one or more second features of the second message and one or more second cluster features of one or more second clusters to which the second message is assigned;
training a message classifier to use a learned decision rule that is trained using a set of training message data labeled based upon one or more spam indicators;
balancing a number of positive spam training message examples with a number of negative non-spam training message examples, in association with the message classifier, by subsampling a ratio of negative non-spam training message examples; and
classifying the second message based upon the one or more message features and the learned decision rule of the message classifier.
19. The non-transitory machine readable medium of claim 18 , wherein the computing cluster features comprises:
measuring a distance between a first bag of words and a second bag of words within a cluster to create a cluster feature.
20. The non-transitory machine readable medium of claim 18 , wherein the operations comprise:
retroactively classifying the second message after the second message was delivered to a message inbox.