Method and apparatus for document analysis and outcome determination
A system for extracting useful information from the documents associated with various courts' dockets uses a bidirectional long short-term memory (BiLSTM)-Attention-conditional random fields (CRF) architecture having a multi-headed output layer producing an outcome and settlement classification. The system includes a BiLSTM layer, an attention layer, a CRF layer, and a sigmoid layer that interact to produce the outcome and settlement classification.
1. An apparatus comprising:
a sequence processing model layer configured to receive text of a document of a case and output an identification of a sequence of words of the text of the document of the case;
an attention layer configured to receive the output of the sequence processing model layer and output data relating to relationships between the words of the text of the document of the case based on the sequence of words of the text of the document of the case;
a conditional random fields (CRF) layer configured to receive the output of the attention layer and output an identification of an entry level outcome based on the output of the attention layer; and
a sigmoid layer configured to receive the output of the attention layer and output an entry level settlement based on the output of the attention layer.
2. The apparatus of claim 1 , wherein the sequence processing model layer comprises a plurality of bidirectional long short-term memory (BiLSTM) components.
3. The apparatus of claim 2 , wherein each of the plurality of BiLSTM components comprises a forward direction long short-term memory (LSTM) and a backward direction LSTM.
4. The apparatus of claim 3 , wherein each of the plurality of BiLSTM components receives input from one of a plurality of smooth inverse frequency (SIF) embedding units.
5. The apparatus of claim 4 , wherein the plurality of SIF embedding units encodes the text of the document and a plurality of one hot encoded entry type units inputs a vector representation to the plurality of SIF embedding units.
6. The apparatus of claim 5 , wherein each of the plurality of one hot encoded entry type units is configured to receive an entry type and convert the entry type into the vector representation.
7. The apparatus of claim 1 , wherein the CRF layer comprises a plurality of CRF blocks.
8. The apparatus of claim 7 , wherein each of the plurality of CRF blocks is in communication with a neighboring CRF and the plurality of CRF blocks determine output of the CRF layer.
9. An apparatus comprising:
a BiLSTM layer comprising a plurality of BiLSTM components each of the plurality of BiLSTM components configured to receive a tensor comprising text of a document of a case;
an attention layer comprising a plurality of attention blocks, each attention block configured to receive outputs of the plurality of BiLSTM components, identify relationships between words of the document of the case, and output an output tensor;
a CRF layer comprising a plurality of CRF blocks, each of the plurality of CRF blocks configured to receive output tensors from each of the plurality of attention blocks and identify elements of the output tensors, the plurality of CRF blocks configured to output an entry level outcome based on the elements of the output tensors; and
a sigmoid layer comprising a plurality of sigmoid blocks, each of the plurality of sigmoid blocks configured to receive an output tensor from one of the plurality of attention blocks and output an entry level settlement.
10. The apparatus of claim 9 , wherein each of the plurality of BiLSTM components comprises a forward direction LSTM and a backward direction LSTM.
11. The apparatus of claim 9 , wherein each of the plurality of BiLSTM components receives input from one of a plurality of SIF embedding units.
12. The apparatus of claim 11 , wherein the plurality of SIF embedding units encodes the text of the document and a plurality of one hot encoded entry type units inputs a vector representation to the plurality of SIF embedding units.
13. The apparatus of claim 12 , wherein each of the plurality of one hot encoded entry type units is configured to receive an entry type and convert the entry type into the vector representation.
14. The apparatus of claim 9 , wherein each of the plurality of CRF blocks is in communication with a neighboring one of the plurality of CRF blocks and the plurality of CRF blocks determine output of the CRF layer.
15. A method comprising:
receiving text of a document of a case at a sequence processing model layer;
outputting an identification of a sequence of words of the text of the document of the case from the sequence processing model layer;
receiving the output of the sequence processing model layer at an attention layer;
outputting an identification of relationships between the words of the text of the case based on the sequence of words of the text of the case, from the attention layer;
receiving the output of the attention layer at a CRF layer;
outputting an identification of an entry level outcome from the CRF layer based on the output of the attention layer;
receiving output of the attention layer at a sigmoid layer; and
outputting an entry level settlement based on the output of the sigmoid layer and the attention layer.
16. The method of claim 15 , wherein the sequence processing model layer comprises a plurality of BiLSTM components.
17. The method of claim 16 , wherein each of the plurality of BiLSTM components comprises a forward direction LSTM and a backward direction LSTM.
18. The method of claim 17 , wherein a first of the plurality of BiLSTM components receives an encoded representation from one of the plurality of BiLSTM components and the other plurality of BiLSTM components receive input from one of a plurality of SIF embedding units.
19. The method of claim 18 , wherein the plurality of SIF embedding units encodes the text of the document and a plurality of one hot encoded entry type units inputs a vector representation to the plurality of SIF embedding units.
20. The method of claim 19 , wherein each of the plurality of one hot encoded entry type units is configured to receive an entry type and convert the entry type into the vector representation.