Method and apparatus for classifying electronic messages
Systems and methods for processing a message are provided. A message may be processed to generate a message summary by removing or replacing certain words, phrases, sentences, punctuation, and the like. Message signatures based upon the message summary may be generated and stored in a signature database, which may be used to identify and/or classify spam messages. Subsequently received messages may be classified by signature and processed based on classification.
1. A method for processing electronic messages (e-mails) is provided, the method comprising:
receiving an electronic message, wherein the received electronic message includes a sender's address;
comparing the sender's address associated with the electronic message to sender addresses found on a blacklist that contains a plurality of sender addresses and signatures of previously sent spam;
executing instructions stored in memory, wherein the instructions are executed by a processor to:
receive user input that classifies the received electronic message when the sender's address for the received electronic message is not found within the blacklist, and
update the blacklist based on the received user input, wherein updating the blacklist comprises:
summarizing content of the received electronic message, wherein summarizing the content of the received electronic message includes identifying that the content includes at least one word that has a plurality of possible canonical equivalents, selecting one of the equivalents for the at least one word based on likely association with the spam, assigning a probability of occurrence to the at least one word in the received electronic message, and the probability of occurrence corresponding to a probability that the at least one word occurs in a language,
generating one or more signatures for the received electronic message based on the summarized content, and
associating the generated signatures with the sender's address, and
process the received electronic message based on the blacklist and based on the selected equivalent most likely being associated with the spam.
2. The method of claim 1 , wherein processing the received message includes blocking the received electronic message when the sender's address is found on the blacklist.
3. The method of claim 1 , wherein the updating further includes spell checking the content of the received electronic message.
4. The method of claim 3 , wherein the spell checking of the content of the received electronic message includes correcting misspelled words and discarding misspelled words that are not correctible.
5. The method of claim 4 , wherein the discarding misspelled words that are not correctible includes exceptions that include URLs and email addresses.
6. The method of claim 1 , wherein summarizing content of the received electronic message includes removing extraneous information from received electronic message.
7. The method of claim 6 , wherein extraneous information that is removed includes a list of stop words, words found in commonly used sentences or phrases by known spammers, punctuations, tabs, and blank lines.
8. The method of claim 1 , wherein summarizing content of the received electronic message includes replacing one or more words found in the received electronic message with the selected canonical equivalent.
9. The method of claim 8 , wherein the replacing of one or more words includes replacing a combination of two or more words with the selected canonical equivalent.
10. The method of claim 8 , wherein summarizing content of the received electronic message includes ranking each of the canonical equivalents based on a probability that each canonical equivalent being used is associated with the spam.
11. The method of claim 1 , wherein generating the signature includes using a transform function on the summarized content.
12. The method of claim 1 , wherein generating the signature requires that the summarized content of the received electronic message be a pre-determined size.
13. A non-transitory computer-readable storage medium having embodied thereon a program executable by a processor for implementing a method for processing electronic messages (e-mails) is provided, the method comprising:
receiving an electronic message, wherein the received electronic message includes a sender's address;
comparing the sender's address associated with the electronic message to sender addresses found on a blacklist that contains a plurality of sender addresses and signatures of previously sent spam,
receiving user input that classifies the received electronic message when the sender's address for the received electronic message is not found within the blacklist,
updating the blacklist based on the received user input, wherein updating the blacklist comprises:
summarizing content of the received electronic message, wherein summarizing the content of the received electronic message includes identifying that the content includes at least one word that has a plurality of possible canonical equivalents, selecting one of the equivalents for the at least one word based on likely association with the spam, and assigning a probability of occurrence to at least one word in the received electronic message, and the probability of occurrence corresponding to a probability that the at least one word occurs in a language,
generating one or more signatures for the received electronic message based on the summarized content, and
associating the generated signatures with the sender's address; and
processing the received electronic message based on the blacklist and based on the selected equivalent most likely being associated with the spam.
14. The non-transitory computer-readable storage medium of claim 13 , wherein processing the received message includes blocking the received electronic message when the sender's address is found on the blacklist.
15. The non-transitory computer-readable storage medium of claim 13 , wherein the updating further includes spell checking the content of the received electronic message.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the spell checking of the content of the received electronic message includes correcting misspelled words and discarding misspelled words that are not correctible.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the discarding misspelled words that are not correctible includes exceptions that include URLs and email addresses.
18. The non-transitory computer-readable storage medium of claim 13 , wherein summarizing content of the received electronic message includes removing extraneous information from received electronic message.
19. The non-transitory computer-readable storage medium of claim 18 , wherein extraneous information that is removed includes a list of stop words, words found in commonly used sentences or phrases by known spammers, punctuations, tabs, and blank lines.
20. An apparatus for processing electronic messages (e-mails) is provided, the apparatus comprising:
a network interface that receives receiving an electronic message, wherein the received electronic message includes a sender's address;
a memory; and
a processor that executes instructions stored in memory, wherein execution of the instructions by the processor:
compares the sender's address associated with the electronic message to sender addresses found on a blacklist that contains a plurality of sender addresses and signatures of previously sent spam,
receives user input that classifies the received electronic message when the sender's address for the received electronic message is not found within the blacklist, and
updates the blacklist based on the received user input, wherein updating the blacklist comprises:
summarizing content of the received electronic message, wherein summarizing the content of the received electronic message includes identifying that the content includes at least one word that has a plurality of possible canonical equivalents, selecting one of the equivalents for the at least one word based on likely association with the spam, assigning a probability of occurrence to at least one word in the received electronic message, and the probability of occurrence corresponding to a probability that the at least one word occurs in a language,
generating one or more signatures for the received electronic message based on the summarized content, and
associating the generated signatures with the sender's address, and
processes the received electronic message based on the blacklist and based on the selected equivalent most likely being associated with the spam.