Multi-stage narrative analysis
An extractive summarization model provides extraction and classification of research assertions, or claims, made by a documented body of research such as a scientific paper or article. Modern electronic publication and dissemination allows tremendous capability for researching and scrutinizing previous documented efforts for further research and study. Accordingly, a substantial volume of material is easily obtained in response to research efforts. The extractive summarization model provides a summarization of this scientific literature by identifying and classifying asserted claims made by a particular research document. Researchers may quickly identify relevant documents based on the extracted claims asserted by the document, facilitating substantive review.
1. A method for multi-stage classification for determining claim assertions of a scientific effort, comprising:
training a first model from an annotated corpus of statements derived from stored documents:
computing, from a narrative representation of a scientific effort, a first group of sentences, based on a comparison of the first model of a first set of annotated features derived from the annotated corpus of statements;
training a second model from the annotated corpus of statements; and
computing, from the first group of sentences, a classification of each of the sentences in the first group of sentences based on comparing the second model of a second set of annotated features derived from the annotated corpus of statements, the second model trained on a subset of the annotated corpus, the subset determined based on the first set of annotated features,
the classification of each sentence defined by a probability having a higher accuracy than a classification based only on a single model derived from both the first set and the second set of annotated features.
2. The method of claim 1 further comprising computing a plurality of groups based on a successive model of a successive set of features trained on a further subset of the corpus of statements.
3. The method of claim 1 wherein:
the first set of annotated features determines a binary designation among a plurality of sentences in the narrative representation; and
the second set of annotated features determines a multiclass designation from the sentences in the binary designation.
4. The method of claim 1 further comprising:
determining the annotated features based on an assigned value of a scientific assertion promoted by each statement of a plurality of statements included in the corpus.
5. The method of claim 1 wherein the first set of annotated features is indicative of statements having a scientific assertion, and the second set of annotated features is indicative of, for the statements indicated as having a scientific assertion, a type of the assertion.
6. A method for multi-stage classification for determining claim assertions of a scientific effort, comprising:
receiving an annotated corpus of statements, the annotations indicative of a scientific assertion proposed by a respective statement in the corpus of statements;
training a model from an annotated corpus of statements derived from stored documents;
computing, from a training narrative of a scientific effort, a group of sentences based on a binary designation determined by the model derived from the annotated corpus; and
computing, from the group of sentences, a plurality of classification groups based on a multiclass designation determined by a model trained from a subset of the annotated corpus, the subset based on the binary designation,
the multiclass designation for each sentence computed as a probability having a higher accuracy than a probability computed from a single model trained from the annotations applied to the entire corpus of statements.
7. The method of claim 6 further comprising:
training the model for binary designations based on annotations designating statements in a corpus as one of either prose or claims to generate a binary classifier model; and
training the model for multiclass designations based on the annotations designating the claims further designating a type of claim to generate a multiclass classifier model.
8. The method of claim 7 further comprising:
receiving a narrative representation representing a production document for classification, the narrative representation defined by a natural language description of the scientific effort defined in a parsable structure including a sequence of sentences; and
performing natural language processing on the narrative representation for generating a hierarchical script representation; and
extracting features from the script representation for comparison with one or more of the binary classifier model and the multiclass classifier model.
9. The method of claim 8 further comprising:
applying the model for binary designation by comparing, for each sentence of a plurality of sentences in the narrative representation, features of the sentences to features of the statements defined by the model for binary designation;
based on the compared features, computing a probability that the sentence is likely to define a scientific assertion; and
concluding, if the probability is greater than a binary classification threshold, that the sentence is a claim.
10. The method of claim 8 further comprising:
applying the model for multiclass designation by comparing, for each sentence designated a claim, features of the sentences to features of the statements defined by the model for multiclass designation;
based on the compared features, computing a respective probability that the sentence is likely to correspond to each type of the type of claim classified by the model for multiclass designation; and
concluding that the sentence is a type of claim corresponding to the highest probability of the respective probabilities.
11. The method of claim wherein 10 associating sentences from the narrative representation based on a correspondence of natural language features with the statements designated as claims.
12. The method of claim 6 further comprising:
determining, from the classification groups, negative examples of sentences for training features indicative of prose, the prose not annotated as a scientific assertion; and
training the respective model on the negative examples.
13. The method of claim 12 further comprising:
identifying segments in the annotated corpus defining scientific assertions;
identifying candidate sentences in the annotated corpus corresponding to the identified segment; and
performing a match to identify a probability that each respective candidate sentence corresponds to the identified segment.
14. A server device having a multi-stage classifier for determining claim assertions of a scientific effort, comprising:
a first stage model responsive to a training narrative of a scientific effort for computing, a group of sentences based on a binary designation, the first model trained from an annotated corpus of statements derived from stored documents;
a second stage model trained from a subset of the annotated corpus of statements, the second stage model responsive to the group of sentences for computing a plurality of classification groups based on a multiclass designation determined from the subset of the group of sentences, the subset based on the binary designation;
the server device configured for executing the first stage model and the second stage model in series; and
a rendering device for receiving the sentences based on the multiclass designation, the multiclass designation for each sentence computed as a probability having a higher accuracy than a probability computed from a single model trained from a single model defining the multiclass designation.
15. The server device of claim 14 , further comprising:
a training corpus based on annotations of a plurality of documents defining a scientific effort,
the server configured for training the first model for binary designations based on annotations designating statements in a corpus as one of either prose or claims to generate a binary classifier model; and
the server configured for training the second model for multiclass designations based on the annotations designating the claims further designating a type of claim to generate a multiclass classifier model.
16. A computer program embodying program code on a non-transitory medium that, when executed by a processor, performs steps for implementing a method for or multi-stage classification for determining claim assertions of a scientific effort, the method comprising:
training a first model from an annotated corpus of statements derived from stored documents;
computing, from a narrative representation of a scientific effort, a first group of sentences, based on a comparison of the first model of a first set of annotated features derived from the annotated corpus of statements;
training a second model from the annotated corpus of statements; and
computing, from the first group of sentences, a classification of each of the sentences in the first group of sentences based on comparing the second model of a second set of annotated features derived from the annotated corpus of statements, the second model trained on a subset of the annotated corpus, the subset determined based on the first set of annotated features,
the classification of each sentence defined by a probability having a higher accuracy than a classification based only on a single model derived from both the first set and the second set of annotated features.
17. A method for multi-stage classification for determining claim assertions of a scientific effort, comprising:
receiving an annotated corpus of statements, the annotations indicative of a scientific assertion proposed by a respective statement in the corpus of statements;
computing, from a training narrative of a scientific effort, a group of sentences based on a binary designation determined by a model derived from the annotated corpus;
training the model for binary designations based on annotations designating statements in a corpus as one of either prose or claims to generate a binary classifier model;
training the model for multiclass designations based on the annotations designating the claims further designating a type of claim to generate a multiclass classifier model; and
computing, from the group of sentences, a plurality of classification groups based on a multiclass designation determined by a model trained from a subset of the annotated corpus, the subset based on the binary designation,
the multiclass designation for each sentence computed as a probability having a higher accuracy than a probability computed from a single model trained from the annotations applied to the entire corpus of statements.