Document-classification system, method and software
Every year, professional classifiers at West Group manually classify over 350,000 headnotes, or abstracts of judicial opinions, across approximately 82,000 separate classes of the Key Number System. Although most headnotes are classified from the memory of the classifiers, a significant number are difficult and thus costly to classify manually. Accordingly, the inventors devised systems, methods, and software that facilitate manual classification of headnotes and documents generally and hard-to-classify headnotes particularly. One exemplary system provides a graphical user interface that concurrently displays an unclassified headnote, a ranked list of one or more candidate classes, a candidate class in combination with adjacent classes of the classification system, and at least one classified headnote that is associated with one of the candidate classes.
1. A method of classifying one or more documents in a classification scheme including two or more classes, with each class having one or more classified document summaries, the method comprising:
summarizing a particular document to define a particular document summary;
automatically generating a list of one or more of the classes, with each listed class having one or more classified document summaries which are similar to the particular document summary, wherein generating a list of one or more of the classes comprises:
defining one or more natural-language or boolean queries based on the particular document summary;
performing one or more searches of the classified document summaries based on one or more of the queries, with one or more of the searches yielding one or more found document summaries;
ranking the one or more found document summaries based on relative similarity to the particular document summary to define one or more ranked document summaries; and
generating the list based on one or more of the ranked document summaries; and
classifying the particular document based on the list of classes.
2. A method of classifying one or more documents in a classification scheme including two or more classes, with each class having one or more classified document summaries, the method comprising:
summarizing a particular document to define a particular document summary;
automatically generating a list of one or more of the classes, with each listed class having one or more classified document summaries which are similar to the particular document summary, wherein generating a list of one or more of the classes comprises:
defining one or more natural-language or boolean queries based on the particular document summary;
performing one or more searches of the classified document summaries based on one or more of the queries, with one or more of the searches yielding one or more found document summaries;
ranking the one or more found document summaries based on relative similarity to the particular document summary to define one or more ranked document summaries; and
generating the list based on one or more of the ranked document summaries; and
classifying the particular document based on the list of classes, wherein classifying the particular document based on the list of classes comprises manually selecting one or more of the classes using a graphical user interface or automatically selecting one or more of the classes using a predetermined selection procedure.
3. The method of claim 2 , wherein:
summarizing comprises a step for summarizing a particular document to define a particular document summary;
automatically generating comprises a step for automatically generating a ranked list of one or more of the classes, with each listed class having one or more classified document summaries which are similar to the particular document summary; and
classifying comprises a step for classifying the particular document based on the list of classes.
4. The method of claim 2 , wherein summarizing a particular document comprises manually summarizing the particular document or electronically summarizing the particular document using a computerized document summarizer.
5. A method of classifying one or more documents, comprising
providing a classification scheme including two or more classes, with each class having one or more classified document summaries logically associated with it;
summarizing a particular document to define a particular document summary;
automatically generating a list of one or more of the classes, with each listed class having one or more classified document summaries which are similar to the particular document summary, wherein generating a list of one or more of the classes comprises:
defining one or more natural-language or boolean queries based on the particular document summary;
performing one or more searches of the classified document summaries based on one or more of the queries, with one or more of the searches yielding one or more found document summaries;
ranking the one or more found document summaries based on relative similarity to the particular document summary to define one or more ranked document summaries; and
generating the list based on one or more of the ranked document summaries; and
classifying the particular document based on the list of classes.
6. A method of classifying one or more documents, comprising
providing a classification scheme including two or more classes, with each class having one or more classified document summaries logically associated with it;
summarizing a particular document to define a particular document summary;
automatically generating a list of one or more of the classes, with each listed class having one or more classified document summaries which are similar to the particular document summary;
classifying the particular document based on the list of classes; and
adding one or more classes to the classification scheme, with each added class having one or more classified document summaries logically associated with it.
7. The method of claim 6 , wherein each class has an associated legal concept and the particular document is a judicial opinion or secondary legal source.
8. The method of claim 6 , wherein the classification scheme conforms at least in part with a version of the West Key Numbering System.
9. A computer-readable magnetic, electronic, or optical medium comprising computer-executable instructions for:
causing a computer to read at least part of a classification scheme into memory, the classification scheme including two or more classes with each class having one or more classified document summaries logically associated with it;
causing the computer to summarize in memory a particular document to define a particular document summary;
causing the computer to generate a list in memory of one or more of the classes, with each listed class having associated with it one or more classified document summaries which are similar to the particular document summary, wherein the instructions for generating a list of one or more of the classes comprises instructions for:
causing the computer to define one or more natural-language or boolean queries based on the particular document summary;
causing the computer to perform one or more searches of the classified document summaries based on one or more of the queries, with one or more of the searches yielding one or more found document summaries;
causing the computer to rank the one or more found document summaries based on relative similarity to the particular document summary to define one or more ranked document summaries; and
causing the computer to generate the list based on one or more of the ranked document summaries; and
causing the computer to classify the particular document based on the list of classes.
10. The medium of claim 9 , further comprising computer-executable instructions for:
causing the computer to request manual input for adding or to automatically add one or more classes to the classification scheme, with each added class having one or more classified document summaries logically associated with it.
11. The medium of claim 9 , wherein the classification scheme conforms at least in part with a version of the West Key Numbering System.