Approximate matching of strings for message filtering
View Patent ↗A method of determining whether a guarded term is represented in a message comprises associating a portion of the message with the guarded term and evaluating a cost of the association. A method of generating a collection of guarded terms that represents an original term comprises generating a plurality of variations of the original term, evaluating similarity of each of the plurality of variations with respect to the original term and determining whether the similarity meets a predetermined criterion.
1. A method of identifying strings in an e-mail message, the method comprising:
receiving an e-mail message; and
executing instructions stored in memory, wherein execution of the instructions by a processor:
identifies a text string in the e-mail message;
determines that the identified text string in the e-mail message is not a safe string, wherein safe strings are predetermined strings stored in a database of acceptable terms and identified as legitimately present in e-mail messages;
associates the text string with a guarded term from a database of guarded terms stored in memory, the guarded term being a string of special interest to a user;
evaluates a cost of the association of the identified text string that dictates a probability that the identified text string is a mutation of the associated guarded term, wherein the evaluation compares similarities and differences between the identified text string and the guarded term, and wherein the evaluation assigns different penalties for the cost based on whether the mutation includes regular characters or special characters;
matches the identified text string with the guarded term when the cost of association of the identified text string meets a predetermined threshold; and
characterizes the e-mail message based on the matching between the identified text string and the guarded term.
2. The method of claim 1 , wherein execution of the instructions further:
determines a plurality of matches between a plurality of text strings in the e-mail message and guarded terms in the database of guarded terms; and
classifies the e-mail message based on the determination of a plurality of matches.
3. The method of claim 1 , wherein execution of the instructions further provides the text string to the database of guarded terms.
4. The method of claim 1 , wherein determining that the identified text string in the e-mail message is not a safe string includes matching the identified text string to a term in a dictionary.
5. The method of claim 1 , wherein determining that the identified text string in the e-mail message is not a safe string includes matching the identified text string to a term in the database of acceptable terms.
6. The method of claim 5 , wherein execution of the instructions further provides a new acceptable term designated by a user to the database of acceptable terms.
7. The method of claim 1 , wherein associating the text string with the guarded term further includes identifying that the text string includes a token equivalent to content in the guarded term.
8. The method of claim 7 , wherein identifying that the text string includes an equivalent token further includes identifying that a location of the token in the text string corresponds to a location of equivalent content in the guarded term.
9. The method of claim 7 , wherein execution of the instructions further replaces the token in the text string with a standard token.
10. The method of claim 1 , wherein evaluating a cost of the association further includes determining a number of mutations required to change the text string into the guarded term.
11. The method of claim 1 , wherein evaluating a cost of the association further includes weighting the number of mutations based on a probability of mutation.
12. The method of claim 11 , wherein execution of the instructions further determines the probability of the mutations based on a letter frequency table.
13. A system for identifying strings in an e-mail message, the system comprising:
a processor; and
memory storing a database of guarded terms and executable instructions, wherein the guarded terms are strings of special interest to a user, and wherein execution of the instructions by the processor:
identifies a text string in the e-mail message,
determines that the identified text string in the e-mail message is not a safe string, wherein safe strings are predetermined strings stored in a database of acceptable terms and identified as legitimately present in e-mail messages,
associates the text string with a guarded term from the database of guarded terms,
evaluates a cost of the association of the identified text string that dictates a probability that the identified text string is a mutation of the associated guarded term, wherein the evaluation compares similarities and differences between the identified text string and the guarded term, and wherein the evaluation assigns different penalties for the cost based on whether the mutation includes regular characters or special characters,
matches the identified text string with the guarded term when the cost of association of the identified text string meets a predetermined threshold, and
characterizes the e-mail message based on the matching between the identified text string and the guarded term.
14. The system of claim 13 , wherein execution of the instructions further:
determines a plurality of matches between a plurality of text strings in the e-mail message and guarded terms in the database of guarded terms; and
classifies the e-mail message based on the determination of a plurality of matches.
15. The system of claim 13 , wherein execution of the instructions further identifies that the text string includes a token equivalent to content in the guarded term.
16. The system of claim 15 , wherein execution of the instructions further identifies that a location of the token in the text string corresponds to a location of equivalent content in the guarded term.
17. The system of claim 13 , wherein the execution of the instructions further determines a number of mutations required to change the text string into the guarded term.
18. The system of claim 17 , wherein execution of the instructions further weighs the number of mutations based on a probability of mutation.
19. A non-transitory computer-readable storage medium, having embodied thereon a program, the program being executable by a processor to perform a method for identifying strings in an e-mail message, the method comprising:
identifying a text string in the e-mail message;
determining that the identified text string in the e-mail message is not a safe string, wherein safe strings are predetermined strings stored in a database of acceptable terms and identified as legitimately present in e-mail messages;
associating the text string with a guarded term from a database of guarded terms stored in memory, the guarded term being a string of special interest to a user;
evaluating a cost of the association of the identified text string that dictates a probability that the identified text string is a mutation of the associated guarded term, wherein the evaluation compares similarities and differences between the identified text string and the guarded term, and wherein the evaluation assigns different penalties for the cost based on whether the mutation includes regular characters or special characters;
matching the identified text string with the guarded term when the cost of association of the identified text string meets a predetermined threshold; and
characterizing the e-mail message based on the matching between the identified text string and the guarded term.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the method further comprises:
determining a plurality of matches between a plurality of text strings in the e-mail message and guarded terms in the database of guarded terms; and
classifying the e-mail message based on the determination of a plurality of matches.