Search expression generation
Methods and systems for generating a search expression. The system begins with an empty search expression, and iteratively expands the search expression until some terminating condition is reached.
1. A method for generating a search expression, the method comprising:
performing, by one or more processors executing instructions stored on a memory:
receiving, via an interface, a selection of a training set of text strings, wherein at least some samples in the training set include instances of a field to be matched by the search expression delimited by a start of the field or an end of the field;
generating an initial search expression as the search expression;
performing an iterative process, wherein each iteration includes:
inferring at least one of a positive character set, a negative character set, or a quantifier to add to the search expression based on the samples in the training set;
expanding the search expression by adding the at least one of a positive character set, a negative character set, or a quantifier to the search expression;
evaluating the search expression after the expansion to determine how well the search expression matches the field in the training set; and
rejecting the search expression if the search expression matches the field less accurately than the search expression prior to the expansion; and
terminating the iterative process when at least one terminating condition is reached.
2. The method of claim 1 wherein the at least one terminating condition includes at least one of a generated search expression matching all fields of the at least one training set or a maximum number of iterations.
3. The method of claim 1 wherein the training set includes samples with no instances of the field that act as counter examples.
4. The method of claim 1 wherein the iterative process implements A* algorithm to search for an optimal search expression.
5. The method of claim 1 wherein the search expression is a regular expression and expanding the search expression comprises expanding the regular expression to match one or more additional characters in the field.
6. The method of claim 1 wherein evaluating the expanded search expression includes evaluating the expanded search expression based on at least one of its length, complexity, generalization, or computational effectiveness.
7. The method of claim 6 wherein the length of the expanded search expression is shorter than the length of the search expression prior to expansion.
8. The method of claim 1 further comprising receiving, using a user interface, instructions regarding a desired search expression from a user.
9. The method of claim 1 further comprising receiving instructions to selecting at least one expanded search expression using a user interface.
10. A system for generating a search expression, the system comprising:
one or more processors with associated memory storing instructions that are executable to cause the one or more processors to:
receive, via an interface, a selection of a training set of a text strings, wherein at least some samples in the training set include instances of a field to be matched by the search expression delimited by a start of the field or an end of the field;
generate an initial search expression as the search expression;
perform an iterative process, wherein each iteration includes to:
infer at least one of a positive character set, a negative character set, or a quantifier to add to the search expression based on the samples in the training set;
expand the search expression by adding the at least one of a positive character set, a negative character set, or a quantifier to the search expression;
evaluate the search expression after the expansion to determine how well the search expression matches the field in the training set; and
reject the search expression if the search expression matches the field less accurately than the search expression prior to the expansion; and
terminate the iterative process when at least one terminating condition is reached.
11. The system of claim 10 wherein the at least one terminating condition includes at least one of a generated search expression matching all fields of the at least one training set or a maximum number of iterations.
12. The system of claim 10 wherein the training set includes samples with no instances of the field that act as counter examples.
13. The system of claim 10 wherein the iterative process implements A* algorithm to search for an optimal search expression.
14. The system of claim 10 wherein the search expression is a regular expression and the regular expression is expanded to match one or more additional characters in the field.
15. The system of claim 10 wherein the evaluation of the expanded search expression is based on at least one of its length, complexity, generalization, or computational effectiveness.
16. The system of claim 15 wherein the length of the expanded search expression is shorter than the length of the search expression prior to expansion.
17. The system of claim 10 further comprising a user interface configured to receive instructions regarding a desired search expression from a user.
18. The system of claim 10 further comprising a user interface configured to receive instructions to select at least one expanded search expression.
19. The method of claim 1 wherein generating the initial search expression comprises generating the initial search expression from the training set.
20. The system of claim 10 wherein the initial search expression is generated from the training set.