Systems and methods for updating predictive coding for document category review
Systems and methods for analyzing documents are provided herein. A plurality of documents and user input are received via a computing device. The user input includes hard coding of a subset of the plurality of documents, based on an identified subject or category. Instructions stored in memory are executed by a processor to generate an initial control set, analyze the initial control set to determine at least one seed set parameter, automatically code a first portion of the plurality of documents based on the initial control set and the seed set parameter associated with the identified subject or category, analyze the first portion of the plurality of documents by applying an adaptive identification cycle, and retrieve a second portion of the plurality of documents based on a result of the application of the adaptive identification cycle test on the first portion of the plurality of documents.
1 . A method of predictive coding for document category review comprising:
receiving a coding category that includes one of responsiveness, issues, and privileges;
receiving training documents that comprise an initial set of review documents;
generating a coded control set of data based on the training documents using coding determinations for the coded control set, the coding determinations performed using a predictive coding of the initial set of review documents;
training machine learning for the coding category utilizing an adaptive identification cycle trained with the initial set of review documents;
coding additional review documents using the coded control set;
presenting a subject document from the additional review documents to a human reviewer;
receiving a correction from the human reviewer to correct at least a portion of the predictive coding by performing a hard coding correction on the subject document, the hard coding correction comprising updating the predictive coding;
receiving the hard coding correction and updating the coded control set with the subject document having the hard coding correction;
applying the updated coded control set to another set of additional documents to predictively code the another set of additional documents;
identifying similar documents to the initial set of review documents utilizing the machine learning to detect concepts within the initial set of review documents;
updating the machine learning by utilizing the adaptive identification cycle with the additional review documents until a confidence threshold validation is met, the confidence threshold validation calculated using the correction from the human reviewer of the subject document and the identifying of the similar documents by the machine learning; and
supplementing the initial set of review documents with the similar documents.
2 . The method according to claim 1 , further comprising applying the coding determinations of the coded control set to contextually similar data in a corpus of documents.
3 . The method according to claim 1 , further comprising receiving a review of each document in the updated coded control set to ensure that proper coding has been implemented.
4 . The method according to claim 1 , wherein generating the coded control set of data comprises generating the coded control set of data based on a plurality of uncoded documents from a corpus, the plurality of uncoded documents comprising documents that are relevant or selected randomly.
5 . The method according to claim 4 , wherein a portion of the initial set of review documents are randomly sampled from an un-reviewed document population.
6 . The method according to claim 1 , further comprising receiving a determination if the subject document coded using the coded control set was miscoded, prior to the step of receiving the hard coding correction to the subject document.
7 . The method according to claim 1 , further comprising updating the training documents with each new coded document created using the coded control set.
8 . A system for predictive coding for document category review, the system comprising:
a processor; and
a memory for storing instructions, the processor executing the instructions to:
receive a coding category that includes one of responsiveness, issues, and privileges;
receive training documents that comprise an initial set of review documents;
generate a coded control set of data based on the training documents using coding determinations for the coded control set, the coding determinations performed using a predictive coding of the initial set of review documents;
train machine learning for the coding category utilizing an adaptive identification cycle trained with the initial set of review documents;
code additional review documents using the coded control set;
present a subject document from the additional review documents to a human reviewer;
receive a correction from the human reviewer to correct at least a portion of the predictive coding by performing a hard coding correction on the subject document, the hard coding correction comprising updating the predictive coding;
receive the hard coding correction and updating the coded control set with the subject document having the hard coding correction;
apply the updated coded control set to another set of additional documents to predictively code the another set of additional documents;
identify similar documents to the initial set of review documents utilizing the machine learning to detect concepts within the initial set of review documents;
update the machine learning by utilizing the adaptive identification cycle with the additional review documents until a confidence threshold validation is met, the confidence threshold validation calculated using the correction from the human reviewer of the subject document and the identifying of the similar documents by the machine learning; and
supplement the initial set of review documents with the similar documents.
9 . The system according to claim 8 , further comprising applying the coding determinations of the coded control set to contextually similar data in a corpus of documents.
10 . The system according to claim 8 , wherein the processor is configured to receive a review of each document in the updated coded control set to ensure that proper coding has been implemented.
11 . The system according to claim 8 , wherein the processor is configured to generate the coded control set of data by generating the coded control set of data based on a plurality of uncoded documents from a corpus, the plurality of uncoded documents comprising documents that are relevant or selected randomly.
12 . The system according to claim 11 , wherein a portion of the initial set of review documents are randomly sampled from an un-reviewed document population.
13 . The system according to claim 8 , wherein the processor is configured to receive a determination if the subject document coded using the coded control set was miscoded, prior to the step of receiving the hard coding correction to the subject document.
14 . The system according to claim 8 , wherein the processor is configured to update the training documents with each new coded document created using the coded control set.
15 . A method for document category review, the method comprising:
receiving a coding category that includes one of responsiveness, issues, and privileges;
generating a coded control set of data based on received training documents using coding determinations for the coded control set, the coding determinations performed using a predictive coding of an initial set of review documents;
training machine learning for the coding category utilizing an adaptive identification cycle trained with the initial set of review documents;
coding additional review documents using the coded control set;
presenting a subject document from additional review documents to a human reviewer, the additional review documents being generated from the coded control set;
receiving a correction from the human reviewer to correct at least a portion of the predictive coding, the correction comprising a hard coding correction on the subject document, the hard coding correction being used to update the predictive coding by updating the coded control set with the subject document having the hard coding correction;
applying the updated coded control set to another set of additional documents to predictively code the another set of additional documents;
identifying similar documents to the initial set of review documents utilizing the machine learning to detect concepts within the initial set of review documents; and
updating the machine learning by utilizing the adaptive identification cycle with the additional review documents until a confidence threshold validation is met, the confidence threshold validation calculated using the correction from the human reviewer of the subject document and the identifying of the similar documents by the machine learning.
16 . The method according to claim 15 , further comprising receiving a determination if the subject document coded using the coded control set was miscoded, prior to the step of receiving the correction.
17 . The method according to claim 16 , further comprising supplementing the initial set of review documents with the similar documents.
18 . The method according to claim 15 , wherein the additional review documents are selected based on comparison of documents with an initial set of relevant documents.
19 . The method according to claim 15 , further comprising automatically coding a document using the coded control set of data.
20 . The method according to claim 19 , further comprising providing the document to a reviewer if the document is incorrectly coded, prior to the step of receiving a hard coding correction to the document.