Systems and methods for message threat management
View Patent ↗The present invention is directed to systems and methods for detecting unsolicited and threatening communications and communicating threat information related thereto. Threat information is received from one or more sources; such sources can include external security databases and threat information data from one or more application and/or network layer security systems. The received threat information is reduced into a canonical form. Features are extracted from the reduced threat information; these features in conjunction with configuration data such as goals are used to produce rules. In some embodiments, these rules are tested against one or more sets of test data and compared against the same or different goals; if one or more tests fail, the rules are refined until the tests succeed within an acceptable margin of error. The rules are then propagated to one or more application layer security systems.
1. A method to classify communications from messaging entities, the method comprising:
initiating a plurality of interrogation engines, each interrogation engine implementing a message classification technique;
initiating a corresponding plurality of index queues, each index queue associated with one interrogation engine;
storing, in each index queue, indexes that index communications in a order in which its corresponding interrogation engine is to process the communications;
receiving a communication from a messaging entity, wherein the communication is a legitimate e-mail message or spam or a virus or a communication that violates corporate policy;
assigning an index to the communication;
performing a load evaluation in response to receiving the communication;
determining an additional interrogation engine should be initiated based on the load evaluation, and in response to the determination, creating a new index queue and initiating a new interrogation engine, wherein the new index queue is associated with the new interrogation engine;
placing the index assigned to the communication into one or more index queues of the plurality of index queues and the new index queue;
for each of the one or more index queues into which the index is assigned, using the corresponding interrogation engine and message classification technique to classify the communication;
combining, using one or more data processors, results of the message classification techniques to generate a message profile score for the communication; and
wherein the message profile score is used in deciding what action is to be taken with respect to the communication associated with the messaging entity, and the communication is interrogated by a plurality of interrogation engines of different types.
2. The method of claim 1 , wherein the communication is an e-mail message or VoIP communication or Instant Messaging communication or SMS message or MMS message.
3. The method of claim 1 , wherein the message classification techniques include at least two techniques selected from the group: Real-time Black-hole Lists (RBLs) classification technique, reputation server classification technique, signature-based classification technique, fingerprinting-based classification technique, message header analysis classification technique, sender authentication set of classification techniques, statistical classification techniques, and content filtering classification technique.
4. The method of claim 1 , wherein each message classification technique is associated with a confidence value of a classification of the communication by the message classification technique, which is used in generating a message classification result from the message classification technique.
5. The method of claim 4 , wherein a classification value from each of the message classification techniques is multiplied by its associated confidence value in order to generate the message classification result.
6. The method of claim 5 , further comprising: iterating through the message classification techniques and allowing each technique to attempt to classify the message; wherein the result of each classification is a numeric value, textual value, or categorical value.
7. The method of claim 1 , wherein combining results of the message classification techniques to generate a message profile score comprises summing together probabilities of each classification technique that the message is unwanted.
8. The method of claim 7 , wherein combining results of the message classification techniques to generate a message profile score comprises summing together probabilities of each classification technique that the message is legitimate.
9. The method of claim 1 , wherein at least one of the message classification techniques includes a reputation scoring technique; and
wherein the reputation scoring technique assigns a reputation probability to a messaging entity;
wherein the reputation probability indicates reputability of a messaging entity based upon an extent to which the communication's characteristics exhibit or conform to one or more reputation-related criteria.
10. The method of claim 1 , wherein the communication was sent over a network.
11. The method of claim 1 , wherein the message profile score is an aggregation of the results of each of the message classification techniques.
12. One or more computer readable media storing instructions that are executable by one or more data processors, and upon such execution cause the one or more data processors to perform operations comprising:
initiating a plurality of interrogation engines, each interrogation engine implementing a message classification technique;
initiating a corresponding plurality of index queues, each index queue associated with one interrogation engine;
storing, in each index queue, indexes that index communications in a order in which its corresponding interrogation engine is to process the communications;
receiving a communication that was sent over a network from a messaging entity, wherein the communication is a legitimate e-mail message or spam or a virus or a communication that violates corporate policy;
assigning an index to the communication;
performing a load evaluation in response to receiving the communication;
determining an additional interrogation engine should be initiated based on the load evaluation, and in response to the determination, creating a new index queue and initiating a new interrogation engine, wherein the new index queue is associated with the new interrogation engine;
placing the index assigned to the communication into one or more index queues of the plurality of index queues and the new index queue; for each of the one or more index queues into which the index is assigned, using the corresponding interrogation engine and message classification technique to classify the communication;
wherein each message classification technique is associated with a confidence value which is used in generating a message classification output from the message classification technique;
combining results of the message classification techniques to generate a message profile score for the communication; and
wherein the message profile score is used in deciding what action is to be taken with respect to the communication associated with the messaging entity, and the communication is interrogated by a plurality of interrogation engines of different types.
13. The computer readable media of claim 12 , wherein the communication is an e-mail message or VoIP communication or Instant Messaging communication.
14. The computer readable media of claim 12 , wherein the profile score is used in determining the communication is a legitimate message or unwanted communication or a communication violative of a pre-selected policy.
15. The computer readable media of claim 14 , wherein an unwanted communication includes a spam or virus communication;
wherein the pre-selected policy includes a corporate communication policy, a messaging policy, a legislation or regulatory policy, or an international communication policy.
16. The computer readable media of claim 12 , wherein the message classification techniques include at least two techniques selected from the group: real-time black-hole lists (RBLs) classification technique, reputation server classification technique, signature-based classification technique, fingerprinting-based classification technique, message header analysis classification technique, sender authentication set of classification techniques, statistical classification techniques, and content filtering classification technique.
17. The computer readable media of claim 12 , wherein the message classification output of is a numeric value, textual value, or categorical value.
18. The computer readable media of claim 12 , wherein at least one of the message classification techniques includes a reputation scoring technique; and
wherein the reputation scoring technique assigns a reputation probability to a messaging entity;
wherein the reputation probability indicates reputability of a messaging entity based upon an extent to which the communication's characteristics exhibit or conform to one or more reputation-related criteria.