Automated tagging of documents

1. An apparatus comprising:

a processor;

a storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

semantic tagging logic executed by the processor for causing generation of at least one first association that associates at least one standard tag with at least one first text fragment of a user-defined document, such that the semantic tagging logic is trained on a standard document annotated with at least one standard tag;

tag association logic executed by the processor for associating at least one first user-defined tag with at least one second text fragment of the user-defined document in response to the at least one second text fragment matching a value associated with the at least one first user-defined tag, to establish at least one second association; and

mapping logic executed by the processor for establishing a mapping between the at least one standard tag and the at least one first user-defined tag in response to existence of a requisite correlation between the at least one standard tag and the at least one first user-defined tag.

2. The apparatus of claim 1 , wherein the requisite correlation exists if a number of times the at least one standard tag and the at least one first user-defined tag are both associated with a same text fragment is greater than a threshold.

3. The apparatus of claim 1 , wherein the requisite correlation exists if a number of co-occurrences for a given tag is greater than a threshold fraction of a total number of co-occurrences, wherein a co-occurrence exists in response to the at least one standard tag and the at least one first user-defined tag both being associated with a same text fragment of the user-defined document.

4. The apparatus of claim 1 , wherein the requisite correlation exists if a total number of co-occurrences is at least a threshold fraction of a number of occurrences in an occurrence set, wherein the occurrence set comprises occurrences of tags selected from a tag set, and a co-occurrence exists in response to the at least one standard tag and the at least one first user-defined tag both being associated with a same text fragment.

5. The apparatus of claim 1 , wherein the requisite correlation exists if a total number of occurrences of tags selected from a tag set is at least a threshold value, wherein an occurrence exists in response to a standard tag being associated with a text fragment.

6. The apparatus of claim 1 , wherein the requisite correlation exists if:

a number of co-occurrences for a given tag is greater than a threshold fraction of a total number of co-occurrences,

a total number of co-occurrences is at least a threshold fraction of a number of occurrences in an occurrence set, wherein the occurrence set comprises occurrences of tags selected from a tag set, and

a total number of occurrences of tags is at least a threshold value,

wherein a co-occurrence exists in response to the at least one standard tag and the at least one first user-defined tag both being associated with a same text fragment, and

an occurrence exists in response to a standard tag being associated with a text fragment.

7. The apparatus of claim 1 , wherein the semantic tagging logic comprises a statistical tagger.

8. The apparatus of claim 7 , wherein the statistical tagger implements a Hidden Markov Model trained with the average sequence perceptron algorithm.

9. The apparatus of claim 1 , wherein the user-defined document comprises at least a portion of an online encyclopedia.

10. The apparatus of claim 9 , wherein the at least one first user-defined tag comprises a property of the online encyclopedia, and the value associated with the at least one first user-defined tag comprises a value associated with the property of the online encyclopedia.

11. The apparatus of claim 1 , wherein the standard document comprises a collection of hand-annotated documents.

12. The apparatus of claim 1 , wherein the standard document comprises manually-labeled training data for named entity recognition.

13. The apparatus of claim 1 , wherein the standard document comprises a CoNLL English Named Entity Recognition dataset, training data of a financial news collection, or a combination thereof.

14. The apparatus of claim 1 , wherein the at least one standard tag comprises a Person tag, an Organization tag, a Location tag, a Miscellaneous tag, or a combination thereof.

15. The apparatus of claim 1 , further comprising:

text selection logic executed by the processor for selecting from the user-defined document at least one tagged text fragment that is associated with at least one second user-defined tag; and

re-tagging logic executed by the processor for associating the at least one tagged text fragment with at least one standard tag associated by the mapping with the at least one second user-defined tag.

16. The apparatus of claim 15 , further comprising:

re-training logic executed by the processor for providing the at least one tagged text fragment and the at least one standard tag associated by the mapping with the at least one second user-defined tag to the semantic tagging logic as training input.

17. The apparatus of claim 15 , further comprising:

extension logic executed by the processor for extending the at least one tagged text fragment to include adjacent words to produce at least one extended text fragment in response to the adjacent words being no-stop words; and

null tagging logic for associating at least one word of the extended text fragment with a null tag in response to the at least one word being a no-stop word.

18. The apparatus of claim 17 , further comprising:

re-training logic executed by the processor for providing the at least one extended text fragment and the association between the at least one word and the null tag to the tagger as training input.

19. An apparatus comprising:

a processor;

a storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

text selection logic executed by the processor for selecting from a user-defined document at least one tagged text fragment that is associated with at least one user-defined tag, such that the at least one user-defined tag is selected from a mapping table that associates the at least one user-defined tag with at least one standard tag; and

tagger training text input logic executed by the processor for providing the at least one tagged text fragment and at least one standard tag associated by the mapping with the at least one user-defined tag to the tagger as training input.

20. The apparatus of claim 19 , further comprising:

text extension logic executed by the processor for extending the at least one tagged text fragment to include adjacent words, in response to the adjacent words being no-stop words, to produce at least one extended text fragment;

extended text association logic executed by the processor for associating at least one word of the extended text fragment with a null tag in response to the at least one word being a no-stop word; and

tagger training extended text input logic executed by the processor for providing the at least one extended text fragment and the association between the at least one word and the null tag to the tagger as training input.

21. A non-transitory computer-readable storage medium for tangibly storing thereon computer readable instructions for a method comprising:

causing a semantic tagger to generate at least one first association that associates a first tag with a first portion of text,

such that the tagger is operable to select the first portion of text from the first collection of documents, the tagger is further operable to select the first tag from a first tag set, and the tagger is trained using training data that includes a second collection of documents annotated with the first tag set;

generating at least one second association that associates a second tag selected from a second tag set with a second portion of text selected from the first collection in response to the second portion of text matching a value associated with the second tag; and

establishing a mapping between the first tag and the second tag in response to existence of a requisite correlation between the first tag and the second tag.

22. The non-transitory computer-readable storage medium of claim 21 , wherein the requisite correlation exists if a number of co-occurrences for at least one tag is greater than a threshold fraction of a total number of co-occurrences, wherein a co-occurrence exists in response to the first tag and the second tag both being associated with a same text fragment.

23. The non-transitory computer-readable storage medium of claim 21 , further comprising instructions for:

selecting a chunk of text from the first portion of text, wherein the chunk of text includes an entity name;

identifying a user-defined tag associated with the entity name, wherein the user-defined tag is selected from the second set of tags;

identifying a standard tag associated with the user-defined tag by the mapping; and associating the entity name with the standard tag.

24. The non-transitory computer-readable storage medium of claim 23 , further comprising instructions for:

providing the first portion of text and the standard tag to the tagger as training input.

25. The non-transitory computer-readable storage medium of claim 21 , further comprising instructions for:

extending the chunk of text to include at least one adjacent word in response to the at least one adjacent word being a no-stop word, to produce at least one extended chunk of text; and

associating the at least one adjacent word with a null tag.

26. The non-transitory computer-readable storage medium of claim 25 , further comprising instructions for: providing the at least one extended chunk of text and the association between the at least one adjacent word and the null tag to the tagger as training input.

27. A method comprising:

causing, using a computing device, a semantic tagger to generate at least one first association that associates a first tag with a first portion of text,

such that the tagger is operable to select the first portion of text from a first collection of documents, and the tagger is further operable to select the first tag from a first tag set, and the tagger is trained using training data that includes a second collection of documents annotated with the first tag set;

generating, using the computing device, at least one second association that associates a second tag selected from a second tag set with a second portion of text selected from the first collection in response to the second portion of text matching a value associated with the second tag; and

establishing, using the computing device, a mapping between the first tag and the second tag in response to existence of a requisite correlation between the first tag and the second tag.

28. The method of claim 27 , wherein the requisite correlation exists if a number of times the at least one standard tag and the at least one user-defined are both associated with a same text fragment is greater than a threshold.

29. The method of claim 27 , further comprising:

selecting, using the computing device, a chunk of text from the first portion of text, wherein the chunk of text includes an entity name;

identifying, using the computing device, a user-defined tag associated with the entity name, wherein the user-defined tag is selected from the second set of tags;

identifying, using the computing device, a standard tag associated with the user-defined tag by the mapping; and

associating, using the computing device, the entity name with the standard tag.

30. The method of claim 29 , further comprising:

providing, using the computing device, the first portion of text and the standard tag to the tagger as training input.

31. The method of claim 30 , further comprising:

extending, using the computing device, the chunk of text to include at least one adjacent word in response to the at least one adjacent word being a no-stop word, to produce at least one extended chunk of text; and

associating the at least one adjacent word with a null tag.

32. The method of claim 31 , further comprising:

providing, using the computing device, the at least one extended chunk of text and the association between the at least one adjacent word and the null tag to the tagger as training input.