Online adaptive filtering of messages
View Patent ↗In general, a two or more stage spam filtering system is used to filter spam in an e-mail system. One stage includes a global e-mail classifier that classifies e-mail as it enters the e-mail system. The parameters of the global e-mail classifier generally may be determined by the policies of e-mail system owner and generally are set to only classify as spam those e-mails that are likely to be considered spam by a significant number of users of the e-mail system. Another stage includes personal e-mail classifiers at the individual mailboxes of the e-mail system users. The parameters of the personal e-mail classifiers generally are set by the users through retraining, such that the personal e-mail classifiers are refined to track the subjective perceptions of their respective user as to what e-mails are spam e-mails.
1. A method of operating a spam filtering system in a messaging system that includes a message gateway and individual message boxes for users of the system, the method comprising the following operations performed by one or more processors:
aggregating personal retraining data used to retrain personal, scoring e-mail classifiers that classify messages delivered to the individual message boxes as spam when a score for the messages exceeds a first threshold for classifying the messages as spam, wherein personal retraining data for an individual message box is based on a user's feedback about the classes of messages in the user's individual message box;
determining a difference between a probability measure calculated for a message in the personal retraining data for the individual message box by a global, scoring e-mail classifier and a second threshold for classifying the messages as spam, the second threshold being higher than the first threshold;
selecting a subset of the aggregated personal retraining data as global retraining data for retraining the global, scoring e-mail classifier that classifies messages received at a message gateway as spam when the difference exceeds a difference threshold; and
retraining the global, scoring e-mail classifier based on the global retraining data so as to adjust which messages are classified as spam.
2. The method of claim 1 wherein the user feedback is explicit.
3. The method of claim 2 wherein the explicit user feedback comprises one or more of the following: a user reporting a message as spam; moving a message from an Inbox folder in the individual message box to a Spam folder in the individual message box; or moving a message from an Spam folder in the individual message box to a Inbox folder in the individual message box.
4. The method of claim 1 wherein the feedback is implicit.
5. The method of claim 4 wherein the implicit feedback comprises one or more of the following: keeping a message as new after the message has been read; forwarding a message; replying to a message; printing a message; adding a sender of a message to an address book; or not explicitly changing a classification of a message.
6. The method of claim 1 wherein the aggregated personal retraining data comprises messages.
7. The method of claim 1 wherein the feedback comprises changing a message's class.
8. The method of claim 7 wherein selecting a subset of the aggregated personal retraining data comprises selecting a message as global retraining data when a particular number of users change the message's classification.
9. The method of claim 1 wherein the messages are e-mails.
10. The method of claim 1 wherein the messages are e-instant messages.
11. The method of claim 1 wherein the messages are SMS messages.
12. The method of claim 1 wherein, to classify a message, the global, scoring e-mail classifier uses an internal model to determine a probability measure for the message and compares the probability measure to a classification threshold.
13. The method of claim 12 wherein, to classify a message, the personal, scoring e-mail classifier uses an internal model to determine a probability measure for the message and compares the probability measure to a classification threshold, the method further comprising initializing the personal, scoring e-mail classifier's internal model using the internal model for the global, scoring e-mail classifier.
14. A non-transitory computer-usable medium storing a computer program for operating a spam filtering system in a messaging system that includes a message gateway and individual message boxes for users of the system, the computer program comprising instructions for causing at least one processor to:
aggregate personal retraining data used to retrain personal, scoring e-mail classifiers that classify messages delivered to the individual message boxes as spam when a score for the messages exceeds a first threshold for classifying the messages as spam, wherein personal retraining data for an individual message box is based on a user's feedback about the classes of messages in the user's individual message box;
determine a difference between a probability measure calculated for a message in the personal retraining data for the individual message box by a global, scoring e-mail classifier and a second threshold for classifying the messages as spam, the second threshold being higher than the first threshold;
select a subset of the aggregated personal retraining data as global retraining data for retraining the global, scoring e-mail classifier that classifies messages received at a message gateway as spam when the difference exceeds a difference threshold; and
retrain the global, scoring e-mail classifier based on the global retraining data so as to adjust which messages are classified as spam.
15. The medium of claim 14 wherein the user feedback is explicit.
16. The medium of claim 15 wherein the explicit user feedback comprises one or more of the following: a user reporting a message as spam; moving a message from an Inbox folder in the individual message box to a Spam folder in the individual message box; or moving a message from an Spam folder in the individual message box to a Inbox folder in the individual message box.
17. The medium of claim 14 wherein the feedback is implicit.
18. The medium of claim 17 wherein the implicit feedback comprises one or more of the following: keeping a message as new after the message has been read; forwarding a message; replying to a message; printing a message; adding a sender of a message to an address book; or not explicitly changing a classification of a message.
19. The medium of claim 14 wherein the aggregated personal retraining data comprises messages.
20. The medium of claim 14 wherein the feedback comprises changing a message's class.
21. The medium of claim 20 wherein to select a subset of the aggregated personal retraining data, the computer program further comprises instructions for causing a processor to select a message as global retraining data when a particular number of users change the message's classification.
22. The medium of claim 14 wherein the messages are e-mails.
23. The medium of claim 14 wherein the messages are e-instant messages.
24. The medium of claim 14 wherein the messages are SMS messages.
25. The medium of claim 14 wherein, to classify a message, the global, scoring e-mail classifier uses an internal model to determine a probability measure for the message and compares the probability measure to a classification threshold.
26. An apparatus for operating a spam filtering system in a messaging system that includes a message gateway and individual message boxes for users of the system, the apparatus comprising:
a network interface configured to receive personal retraining data for an individual message box used to retrain personal, scoring e-mail classifiers that classify messages delivered to the individual message boxes as spam when a score for the messages exceeds a first threshold for classifying the messages as spam, wherein the personal retraining data is based on a user's feedback about the classes of messages in the user's individual message box over one or more network connections; and
at least one processor configured by a set of instructions to (i) aggregate the received personal retraining data,
(ii) determine a difference between a probability measure calculated for a message in the personal retraining data for the individual message box by a global, scoring e-mail classifier and a second threshold for classifying the messages as spam, the second threshold being higher than the first threshold,
(iii) select a subset of the aggregated personal retraining data as global retraining data for retraining the global, scoring e-mail classifier that classifies messages received at a message gateway as spam when the difference exceeds a difference threshold, and
(iii) retrain the global, scoring e-mail classifier based on the global retraining data so as to adjust which messages are classified as spam.