Privacy-preserving labeling and classification of email
Emails or other communications are labeled with a category label such as “spam” or “good” without using confidential or Personally Identifiable Information (PII). The category label is based on features of the emails such as metadata that do not contain PII. Graphs of inferred relationships between email features and category labels are used to assign labels to emails and to features of the emails. The labeled emails are used as a training dataset for training a machine learning model (“MLM”). The MLM identifies unwanted emails such as spam, bulk email, phishing email, and emails that contain malware.
1. A method of labeling email, the method comprising:
receiving an unlabeled email;
identifying a feature and a second feature of the unlabeled email, wherein the feature and the second feature do not include personally identifiable information (“PII”) in a body of the unlabeled email;
receiving a labeled cluster comprising an email-category label and seed data;
assigning the email-category label to the unlabeled email based on a first derivative edge in an expansion graph thereby creating a labeled email, wherein the first derivative edge is a directional edge from the labeled cluster to the feature and the first derivative edge represents first inference logic that the email-category label associated with the labeled cluster is also associated with the unlabeled email;
including the labeled email in a training dataset;
assigning the email-category label to the second feature based on a clustering edge in the expansion graph, wherein the clustering edge is a directional edge from the feature to the second feature and the clustering edge represents second inference logic that a label associated with the feature is also associated with the second feature;
assigning the email-category label to a second unlabeled email based on a second derivative edge in the expansion graph thereby creating a second labeled email, wherein the second derivative edge is a directional edge from the second feature to the second unlabeled email and the second derivative edge represents third inference logic that the email-category label associated with the second feature is also associated with the second unlabeled email;
including the second labeled email in the training dataset;
training a machine learning model to classify email with the training dataset; and
classifying a received email with the machine learning model as spam; and
quarantining the received email on a server without downloading to a local computer.
2. The method of claim 1 , wherein the feature comprises sender email address, sender email host server, timestamp, a universal resource locator included in the email, or a hash of at least a portion of the body of the unlabeled email.
3. The method of claim 1 , wherein the email-category label is good email, spam email, phishing email, bulk email, or malware email and the method further comprises: moving the received email to a folder or deleting the received email based on the email-category label.
4. The method of claim 1 , wherein the seed data comprise a previously labeled email, a denylist, an allowlist, or a communication graph.
5. The method of claim 1 , wherein the assigning the email-category label is based on the feature belonging to the labeled cluster.
6. The method of claim 5 , wherein there is a confidence level representing a likelihood that the email-category label is an accurate label for the labeled cluster.
7. The method of claim 1 , further comprising:
identifying a third feature of the unlabeled email;
receiving a second labeled cluster comprising a second email-category label and second seed data;
assigning the second email-category label to the unlabeled email based on the third feature and the second labeled cluster; and
determining that the email-category label has a higher probability of being correct than the second email-category label by applying a set of voting rules.
8. The method of claim 7 , wherein the set of voting rules includes one or more rules for: removing both conflicting email-category labels, retaining an email-category label associated with a highest confidence level, determining that two different email-category labels are not conflicting and retaining both email-category labels, or retaining an email-category label associated with most recent seed data.
9. Computer-readable storage media comprising instructions that when executed cause a computing device to:
identify a feature and a second feature of an unlabeled email, wherein the feature and the second feature do not include personally identifiable information (“PII”) in a body of the unlabeled email;
receive a labeled cluster comprising an email-category label and seed data;
assign the email-category label to the unlabeled email based on a derivative edge in an expansion graph thereby creating a labeled email,
wherein the derivative edge is a directional edge from the labeled cluster to the feature of the unlabeled email and the derivative edge represents first inference logic that the email-category label associated with the labeled cluster is also associated with the unlabeled email;
include the labeled email in a training dataset;
assign the email-category label to a second unlabeled email based on a clustering edge in the expansion graph thereby creating a second labeled email,
wherein the clustering edge is a directional edge from the feature to the second feature and the clustering edge represents second inference logic that a label associated with the feature is also associated with the second feature;
include the second labeled email in the training dataset;
train a machine learning model to classify email with the training dataset; and
classify a received email with the machine learning model as spam; and
quarantine the received email on a server without downloading to a local computer.
10. The computer-readable storage media of claim 9 , wherein the feature comprises sender email address, sender email host server, timestamp, a universal resource locator included in the email, or a hash of at least a portion of the body of the unlabeled email;
wherein the email-category label is good email, spam email, phishing email, bulk email, or malware email and the instructions when executed further cause the computing device to: move the received email to a folder or delete the received email based on the email-category label; and wherein the seed data comprise a previously labeled email, a denylist, an allowlist, or a communication graph.
11. The computer-readable storage media of claim 9 , wherein the instructions further cause the computing device to:
identify a third feature of the unlabeled email;
receive a second labeled cluster comprising a second email-category label and second seed data;
assign the second email-category label to the unlabeled email based on the third feature and the second labeled cluster; and
determine that the email-category label has a higher probability of being correct than the second email-category label by applying a set of voting rules.
12. The computer-readable storage media of claim 11 , wherein the set of voting rules includes one or more rules for: removing both conflicting email-category labels, retaining an email-category label associated with a highest confidence level, determining that two different email-category labels are not conflicting and retaining both email-category labels, or retaining an email-category label associated with most recent seed data.
13. The computer-readable storage media of claim 9 , wherein the feature comprises a hash of at least a portion of the body of the unlabeled email and wherein the instructions further cause the computing device to generate the hash from at least a portion of the body of the unlabeled email by applying a hashing algorithm.
14. A system comprising:
one or more processing units;
one or more memory units coupled to the one or more processing units, the one or more memory units containing instructions that when executed by the one or more processing units cause the system to:
assign an email-category label to an unlabeled email based on a directional, derivative edge to a feature of the unlabeled email from a labeled cluster comprising an email-category label and seed data thereby creating a labeled email, wherein the feature does not include personally identifiable information (“PII”) from a body of the unlabeled email;
assign the email-category label to a second feature of the unlabeled email based on a directional, clustering edge from the unlabeled email to the second feature;
classify email by use of a machine learning model trained on a training dataset that includes the labeled email; and
classify a received email with the machine learning model as spam; and
quarantine the received email on a server without downloading to a local computer.
15. The system of claim 14 , wherein the feature comprises sender email address, sender email host server, timestamp, a universal resource locator included in the unlabeled email, or a hash of at least a portion of the body of the unlabeled email;
wherein the email-category label is good email, spam email, phishing email, bulk email, or malware email and the system is further configured to: move the received email to a folder or delete the received email based on the email-category label; and
wherein the seed data comprise a previously labeled email, a denylist, an allowlist, or a communication graph.
16. The system of claim 14 , wherein the email-category label is assigned based on the feature belonging to the labeled cluster.
17. The system of claim 16 , wherein the instructions further cause the system to assign a confidence level representing a likelihood that the email-category label is an accurate label for the labeled cluster.
18. The system of claim 14 , wherein the instructions further cause the system to assign a second email-category label to the unlabeled email based on a second feature of the unlabeled email and a second labeled cluster comprising a second email-category label and second seed data thereby creating a labeled email labeled with the email-category label and the second email-category label; and
the instructions further cause the system to determine that the email-category label has a higher probability of being correct than the second email-category label by applying a set of voting rules.
19. The system of claim 18 , wherein the set of voting rules includes one or more rules for: removing both conflicting email-category labels, retaining an email-category label associated with a highest confidence level, determining that two different email-category labels are not conflicting and retaining both email-category labels, or retaining an email-category label associated with most recent seed data.