Information extraction and annotation systems and methods for documents

1. A method, comprising:

receiving, by a context analysis module, annotated documents, the annotated documents comprising annotated fields;

analyzing, by the context analysis module, the annotated documents to determine contextual information for each of the annotated fields;

determining discriminative sequences using the contextual information by:

determining, by a contiguity heuristics module, longest contiguous common subsequences between aligned pairs of strings of the annotated documents;

determining, by the contiguity heuristics module, a frequency of occurrence of similar longest contiguous common subsequences; and

wherein the contiguity heuristics module generates a proposed rule from longest contiguous common subsequences having a desired frequency of occurrence;

providing, by the context analysis module, the proposed rule to a document annotator; and

applying, by a rule-based annotator, a rule-based extractor to a target document to create an annotated document, the rule-based extractor being generated from the proposed rule.

2. The method of claim 1 , wherein discriminative sequences are identified within the annotated documents and used to develop both rule-based extractors and feature-based extractors.

3. The method of claim 1 , wherein sequence alignment is utilized to identify discriminative sequences in the context of a specified field.

4. The method of claim 1 , wherein annotation of a document may include associating a word, date, or other group of characters with a particular field.

5. The method of claim 1 , further comprising:

executing, by the context analysis module, a base annotation of original documents to create documents with base annotations, the base annotations comprising basic categories of words or groups of characters; and

providing the documents with base annotations to the document annotator via a user interface.

6. The method of claim 5 , further comprising highlighting each of the base annotations within the user interface.

7. The method of claim 1 , further comprising receiving feedback from the document annotator; and using the feedback to any of approve the proposed rule, modify the proposed rule, and reject the proposed rule.

8. The method of claim 1 , further comprising converting, by a rule-based extractor generator, the proposed rule into the rule-based extractor.

9. The method of claim 1 , wherein determining longest contiguous common subsequences comprises:

aligning pairs of strings having possible contextual matches;

normalizing the pairs of strings by extracting matching segments having a given length; and

aggregating the normalized pairs of strings.

10. The method of claim 9 , further comprising applying a greedy contiguity heuristic to the aggregated normalized pairs of strings.

11. The method of claim 10 , wherein the greedy contiguity heuristic evaluates any of a number of matching segments, a number of gaps between segments, and variances between segment lengths.

12. A system, comprising:

a processor; and

logic encoded in one or more tangible media for execution by the processor, the logic when executed by the processor causing the system to perform operations comprising:

receiving annotated documents comprising annotated fields;

analyzing the annotated documents to determine contextual information for each of the annotated fields;

determining discriminative sequences using the contextual information by:

determining, by a contiguity heuristics module, longest contiguous common subsequences between aligned pairs of strings of the annotated documents;

determining, by the contiguity heuristics module, a frequency of occurrence of similar longest contiguous common subsequences, wherein the contiguity heuristics module generates a proposed rule from longest contiguous common subsequences having a desired frequency of occurrence, wherein the proposed rule is generated using the discriminative sequences, the annotated fields and a feature set; and

providing the proposed rule or the feature set to a document annotator; and wherein a rule-based annotator applies a rule-based extractor to a target document to create an annotated document, the rule-based extractor being generated from the proposed rule.

13. The system of claim 12 , wherein the discriminative sequences are identified within the annotated documents and used to develop both rule-based extractors and feature-based extractors.

14. The system of claim 12 , wherein the processor further executes the logic to perform operations of:

executing a base annotation of original documents to create documents with base annotations, the base annotations comprising basic categories of words or groups of characters; and

providing the documents with base annotations to the document annotator via a user interface.

15. The system of claim 14 , wherein the processor further executes the logic to perform operations of highlighting each of the base annotations within the user interface.

16. The system of claim 12 , wherein the processor further executes the logic to perform operations of receiving feedback from a document annotator; and using the feedback to any of approve the proposed rule, modify the proposed rule, and reject the proposed rule.

17. The system of claim 12 , further comprising a rule-based extractor generator that converts the proposed rule into the rule-based extractor.

18. The system according to claim 17 , further comprising the rule-based annotator that applies the rule-based extractor to a target document to create an annotated document.

19. A method, comprising:

receiving, by a context analysis module, annotated documents, the annotated documents comprising annotated fields;

analyzing, by the context analysis module, the annotated documents to determine contextual information for each of the annotated fields;

determining discriminative sequences using the contextual information by:

determining, by a contiguity heuristics module, longest contiguous common subsequences between aligned pairs of strings of the annotated documents; and

determining, by the contiguity heuristics module, a frequency of occurrence of similar longest contiguous common subsequences, wherein the contiguity heuristics module generates a proposed rule from longest contiguous common subsequences having a desired frequency of occurrence;

generating, by the context analysis module, a feature set using the discriminative sequences and the annotated fields;

providing, by the context analysis module, the feature set to a document annotator; and

applying, by a feature-based annotator, a feature-based extractor to a target document to create an annotated document, the feature-based extractor being generated from the feature set.