On-the-fly pattern recognition with configurable bounds
View Patent ↗Some embodiments of on-the-fly pattern recognition with configurable bounds have been presented. In one embodiment, a pattern matching engine is configured based on user input, which may include values of one or more user configurable bounds on searching. Then the configured pattern matching engine is used to search for a set of features in an incoming string. A set of scores is updated based on the presence of any of the features in the string while searching for the features. Each score may indicate a likelihood of the content of the string being in a category. The search is terminated if the end of the string is reached or if the user configurable bounds are met. After terminating the search, the scores are output.
1. A computer-implemented method comprising:
configuring a pattern matching engine based on user input, the user input comprising one or more user configurable bounds on searching and the one or more user configurable bounds including a maximum barren sample size;
using the pattern matching engine to perform a search for a plurality of features in an incoming string;
updating a plurality of scores based on presence of any of the plurality of features in the string while searching for the plurality of features, wherein each of the plurality of scores indicates a likelihood of content of the string being in one of a plurality of categories;
terminating the search before reaching an end of the string if the one or more user configurable bounds are met;
outputting the plurality of scores after terminating the search;
searching for the plurality of features in a set of pre-classified documents; classifying the set of pre-classified documents based on a set of current scores of each of the pre-classified documents with respect to the plurality of categories while searching for the plurality of features;
collecting statistics on accuracy of said classifying of the set of pre-classified documents; and
generating a user interface based on the statistics collected to receive the user input.
2. The method of claim 1 , wherein generating said user interface includes building a histogram from the statistics collected to indicate a tradeoff between accuracy of classification and at least one of the one or more user configurable bounds.
3. The method of claim 1 , wherein the one or more configurable bounds further comprises at least one of:
a predetermined sample size;
a predetermined search time limit;
a predetermined threshold of a confidence level of the content of the string being in one of the plurality of categories; and
a minimum salience.
4. The method of claim 1 , wherein updating the plurality of scores further comprises:
if a first one of the plurality of features is found in the string, applying a corresponding one of a plurality of sets of weights to update the plurality of scores, wherein each weight of the corresponding one of the plurality of sets of weights is associated with a distinct one of the plurality of scores.
5. The method of claim 1 , further comprising:
classifying the content of the string based on the plurality of scores after terminating the search.
6. The method of claim 5 , further comprising:
blocking the string and causing a client machine to render an error message if the content of the string is classified to be in at least one of a set of one or more predetermined categories; and
causing the client machine to render the content of the string if the content of the string is not classified to be in any of the set of one or more predetermined categories.
7. The method of claim 1 , further comprising:
using the plurality of scores to generate a content classification model usable by content filters to classify content.
8. The method of claim 1 , wherein the plurality of features include a plurality of N-grams.
9. A non-tangible machine-accessible medium that provides instructions that, if executed by a processor, will cause the processor to perform operations comprising:
configuring a pattern matching engine based on user input, the user input comprising one or more user configurable bounds on searching and the one or more user configurable bound including a maximum barren sample size;
using the pattern matching engine to perform a search for a plurality of features in an incoming string;
updating a plurality of scores based on presence of any of the plurality of features in the string while searching for the plurality of features, wherein each of the plurality of scores indicates a likelihood of content of the string being in one of a plurality of categories;
terminating the search before reaching an end of the string if the one or more user configurable bounds are met;
outputting the plurality of scores after terminating the search;
searching for the plurality of features in a set of pre-classified documents; classifying the set of pre-classified documents based on a set of current scores of each of the pre-classified documents with respect to the plurality of categories while searching for the plurality of features;
collecting statistics on accuracy of said classifying of the set of pre-classified documents; and
generating a user interface based on the statistics collected to receive the user input.
10. The non-tangible machine-accessible medium of claim 9 , wherein generating said user interface includes building a histogram from the statistics collected to indicate a tradeoff between accuracy of classification and at least one of the one or more user configurable bounds.
11. The non-tangible machine-accessible medium of claim 9 , wherein the one or more configurable bounds further comprises at least one of:
a predetermined sample size;
a predetermined search time limit;
a predetermined threshold of a confidence level of the content of the string being in one of the plurality of categories; and
a minimum salience.
12. The non-tangible machine-accessible medium of claim 9 , wherein updating the plurality of scores further comprises:
if a first one of the plurality of features is found in the string, applying a corresponding one of a plurality of sets of weights to update the plurality of scores, wherein each weight of the corresponding one of the plurality of sets of weights is associated with a distinct one of the plurality of scores.
13. The non-tangible machine-accessible medium of claim 9 , wherein the operations further comprise:
classifying the content of the string based on the plurality of scores after terminating the search.
14. The non-tangible machine-accessible medium of claim 13 , wherein the operations further comprise:
blocking the string and causing a client machine to render an error message if the content of the string is classified to be in at least one of a set of one or more predetermined categories; and
causing the client machine to render the content of the string if the content of the string is not classified to be in any of the set of one or more predetermined categories.
15. The non-tangible machine-accessible medium of claim 9 , wherein the operations further comprise:
using the plurality of scores to generate a content classification model usable by content filters to classify content.
16. The non-tangible machine-accessible medium of claim 9 , wherein the plurality of features include a plurality of N-grams.
17. A computer-implemented method, comprising:
configuring a pattern matching engine based on user input, the user input comprising one or more user configurable bounds on searching and the one or more user configurable bounds including a maximum barren sample size, and wherein the maximum barren sample size is the maximum size of a subset of the incoming string to be examined if the pattern matching engine fails to find a minimum number of features in the incoming string;
using the pattern matching engine to perform a search for a plurality of features in an incoming string;
updating a plurality of scores based on presence of any of the plurality of features in the string while searching for the plurality of features, wherein each of the plurality of scores indicates a likelihood of content of the string being in one of a plurality of categories;
terminating the search before reaching an end of the string if the one or more user configurable bounds are met; and
outputting the plurality of scores after terminating the search.