Automated document filtration and prioritization for document searching and access
Computer based methods, systems, and computer readable media for classifying documents within a content repository or documents within the document subsets are provided. Documents may be pre-processed to render document sections visible to machine readers. Document subsets may be generated based on user-defined terms. The machine readable documents may be classified within the content repository into one of a group of categories, based-upon the number of times classification terms appear in a specific document section of the document. Documents may be ranked based upon the frequency of classification terms in the specific section. Documents may be associated with specific diseases such as cancer, genes, gene variants, and drugs or synonyms thereof by comparing relevant search terms to specific sections of the documents.
1. A method for classifying documents within a content repository comprising:
pre-processing documents to render document sections visible to machine readers;
generating document subsets from the documents based on user-defined terms;
classifying documents within the content repository or documents within the document subsets into one or more categories, based upon a number of times classification terms appear in a specific section of the document or an article type tag, wherein classifying documents based on the number of times classification terms appear in the specific section of the document comprises applying predefined weighting values to the number of times classification terms appear in the specific section, wherein the predefined weighting values are selected based on a specific number equal to a sum of unique classification terms in the specific section compared to a matching threshold value of a plurality of matching threshold values, wherein each matching threshold value is associated with a specific section of the document;
ranking documents based upon the frequency of classification terms in the specified section; and
associating documents with specific diseases, genes, gene variants, and drugs or synonyms thereof by comparing relevant search terms to specific sections of the documents.
2. The method of claim 1 , wherein the categories are selected from the group consisting of functional, clinical, case reports, reviews, or meetings and proceedings abstracts.
3. The method of claim 1 , wherein the article type tag specifies the category of the document.
4. The method of claim 1 , wherein classification terms are provided by a custom classification terms list, wherein for functional and clinical articles,
when the sum of unique classification terms identified in a section of the document exceeds the matching threshold value, weighting the classification terms equally; and
when the sum of unique classification terms identified in a section of the document is less than the matching threshold value, weighting the classification terms differentially.
5. The method of claim 1 , wherein the specific section used to classify an article as clinical is an abstract section.
6. The method of claim 1 , wherein the specific section used to classify an article as functional is a methods and materials or experimental section.
7. The method of claim 1 , wherein a document is classified as both a clinical and a functional document.
8. A computer system for classifying documents within a content repository wherein the system comprises at least one processor configured to:
pre-process documents to render document sections visible to machine readers;
generating document subsets from the documents based on user-defined terms;
classify documents within the content repository or documents within the document subsets into one or more categories, based upon a number of times classification terms appear in a specific section of the document or an article type tag, wherein classifying documents based on the number of times classification terms appear in the specific section of the document comprises applying predefined weighting values to the number of times classification terms appear in the specific section, wherein the predefined weighting values are selected based on a specific number equal to a sum of unique classification terms in the specific section compared to a matching threshold value of a plurality of matching threshold values, wherein each matching threshold value is associated with a specific section of the document;
rank documents based upon the frequency of classification terms in the specified section; and
associate documents with specific diseases, genes, gene variants, and drugs or synonyms thereof by comparing relevant search terms to specific sections of the documents.
9. The computer system of claim 8 , wherein the categories are selected from the group consisting of functional, clinical, case reports, reviews, or meetings and proceedings abstracts.
10. The computer system of claim 8 , wherein the article type tag specifies the category of the document.
11. The computer system of claim 8 , wherein classification terms are provided by a custom classification terms list, wherein for functional and clinical articles,
when the sum of unique classification terms identified in a section of the document exceeds the matching threshold value, weighting the terms equally; and
when the sum of unique classification terms identified in a section of the document is less than a matching threshold value, weighting the terms differentially.
12. The computer system of claim 8 , wherein the specific section used to classify an article as clinical is an abstract section.
13. The computer system of claim 8 , wherein the specific section used to classify an article as functional is a methods and materials or experimental section.
14. The computer system of claim 8 , wherein a document is classified as both a clinical and a functional document.
15. A computer program product for classifying documents in a content repository, the computer program product comprising one or more computer readable storage media collectively having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
pre-process documents to render document sections visible to machine readers;
generating document subsets from the documents based on user-defined terms;
classify documents within the content repository or documents within the document subsets into one or more categories, based upon a number of times classification terms appear in a specific section of the document or based on an article type tag, wherein classifying documents based on the number of times classification terms appear in the specific section of the document comprises applying predefined weighting values to the number of times classification terms appear in the specific section, wherein the predefined weighting values are selected based on a specific number equal to a sum of unique classification terms in the specific section compared to a matching threshold value of a plurality of matching threshold values, wherein each matching threshold value is associated with a specific section of the document;
rank documents based upon the frequency of classification terms in the specified section; and
associate documents with specific diseases, genes, gene variants, and drugs or synonyms thereof by comparing relevant search terms to specific sections of the documents.
16. The computer program product of claim 15 , wherein the categories are selected from the group consisting of functional, clinical, case reports, reviews, or meetings and proceedings abstracts.
17. The computer program product of claim 15 , wherein the article type tag specifies the category of the document.
18. The computer program product of claim 15 , wherein classification terms are provided by a custom classification terms list, wherein for functional and clinical articles, when the sum of unique classification terms identified in a section of the document exceeds the matching threshold value, weighting the classification terms equally; and when the sum of unique classification terms identified in a section of the document is less than the matching threshold value, weighting the classification terms differentially.
19. The computer program product of claim 15 , wherein the specific section used to classify an article as clinical is an abstract section.
20. The computer program product of claim 15 , wherein the specific section used to classify an article as functional is a methods and materials or experimental section.