Message classification using a summary
View Patent ↗A system and method are disclosed for classifying a message. The method includes receiving a message, summarizing the message to produce a summary, generating a signature based on the summary, and comparing the signature to a database of signatures generated by previously classified messages.
1. A method for classifying a message comprising:
receiving a message, the message including a message body;
summarizing the body of the message to produce a summary, wherein summarizing the body of the message includes removing at least one word that is not to be used in classifying the message, and replacing a word in the body of the message with a canonical equivalent which identifies a group of synonymous words or phrases;
generating a signature based on the summary;
comparing the signature based on the summary to a database of signatures corresponding to signatures of previously received and classified messages;
classifying the message as an unsolicited message if the signature based on the summary matches a signature in the database of signatures, the signature in the database of signatures corresponding to a previously identified unsolicited message; and
transferring the unsolicited message to a storage medium reserved for unsolicited messages.
2. The method of claim 1 , further comprising adding the signature based on the summary to the database of signatures following a manual indication that the received message is an unsolicited message.
3. The method of claim 1 , wherein if the word has a plurality of canonical equivalents, replacing the word in the body of the message with its canonical equivalent includes using a heuristic for selecting a canonical equivalent from the plurality of canonical equivalents.
4. The method of claim 1 , wherein summarizing the body of the message further comprises:
checking the spelling of the body of the message;
correcting the spelling of any words misspelled in the body of the message as identified during the checking of the spelling of the body of the message; and
identifying those words whose spelling cannot be corrected as words that are not to be used in summarizing the body of the message.
5. The method of claim 1 , removing at least one word that is not to be used in classifying the message further comprises removing words not useful for summarizing the body of the message.
6. The method of claim 5 , wherein the identified words are redundant words.
7. The method of claim 1 , wherein summarizing the body of the message further includes sorting words in the body of the message.
8. The method of claim 1 , wherein summarizing the body of the message further includes evaluating an importance of a word in the body of the message.
9. The method of claim 8 , wherein the importance of the word in the body of the message is determined by its probability of occurrence.
10. The method of claim 1 , wherein summarizing the body of the message further includes evaluating an importance of a word in the body of the message and selecting a plurality of important words from a list of canonically equivalent words.
11. The method of claim 1 , wherein summarizing the body of the message to produce a summary further includes using a topic specific summarization technique.
12. The method of claim 1 , wherein summarizing the body of the message to produce a summary further includes using a neural net based summarization technique.
13. The method of claim 1 , wherein generating a signature of the message body based on the summary includes applying a transform function to the summary.
14. The method of claim 1 , wherein generating a signature of the message body based on the summary includes applying a transform function to a word in the summary.
15. The method of claim 1 , wherein the signature is the summary.
16. The method of claim 1 , wherein removing at least one word that is not to be used in classifying the message further comprises discarding a word that is smaller than a predefined size.
17. The method of claim 1 , further comprising identifying distinguishing properties in the body of the message.
18. The method of claim 17 , further comprising generating an additional signature based on the identified distinguishing properties.
19. A system for classifying a message comprising:
a network configured for the distribution of electronic mail messages;
a signature database coupled to the network, the signature database configured to store signatures corresponding to previously received and classified electronic mail messages;
a mail server coupled to the network, the mail server configured to receive an electronic mail message over the network, wherein the mail server includes a summarization and signature generation engine configured to:
summarize the message body of the received electronic mail to produce a summary, wherein summarizing the body of the electronic mail message includes removing at least one word that is not to be used in classifying the message, and replacing a word in the body of the electronic mail message with a canonical equivalent which identifies a group of synonymous words or phrases;
generate a signature based on the summary of the message body;
compare the signature based on the summary to the signatures in the signature database;
classify the received message as an unsolicited message if the signature based on the summary matches a signature in the database of signatures, the signature in the database of signatures corresponding to a previously identified unsolicited message; and
prevent the unsolicited message from being delivered to an electronic-mail client coupled to the mail server via the network by transferring the received message to a storage medium reserved for unsolicited messages.
20. A computer-readable storage medium having embodied thereon a program executable by a processor to perform a method classifying a message, the method comprising:
receiving a message, the message including a message body;
summarizing the body of the message to produce a summary, wherein summarizing the body of the message includes removing at least one word that is not to be used in classifying the message, and replacing a word in the body of the message with a canonical equivalent which identifies a group of synonymous words or phrases;
generating a signature based on the summary of the body;
comparing the signature based on the summary to a database of signatures corresponding to signatures of previously received and classified messages;
classifying the received message as an unsolicited message if the signature based on the summary matches a signature in the database of signatures, the signature in the database of signatures corresponding to a previously identified unsolicited message; and
transferring the unsolicited message to a storage medium reserved for unsolicited messages.