Adaptive probabilistic latent semantic analysis system for automated document coding and review in electronic discovery
Systems and methods for analyzing documents are provided herein. A plurality of documents and user input are received via a computing device. The user input includes hard coding of a subset of the plurality of documents, based on an identified subject or category. Instructions stored in memory are executed by a processor to generate an initial control set, analyze the initial control set to determine at least one seed set parameter, automatically code a first portion of the plurality of documents based on the initial control set and the seed set parameter associated with the identified subject or category, analyze the first portion of the plurality of documents by applying an adaptive identification cycle, and retrieve a second portion of the plurality of documents based on a result of the application of the adaptive identification cycle test on the first portion of the plurality of documents.
1 . A method for document review in electronic discovery, comprising:
receiving from user input, a document review coding category;
generating, by machine assisted automatic coding, based on the document review coding category, an initial control set comprising documents from a corpus of documents;
training probabilistic latent semantic analysis (PLSA) on the initial control set to automatically detect concepts within the documents of the corpus via a statistical analysis of word contexts;
performing the machine assisted automatic coding on the documents of the corpus, by executing the PLSA trained on the initial control set to assign coding determinations to the documents of the corpus according to the document review coding category, based on the PLSA's automatic detection of concepts within the documents of the corpus via the statistical analysis of the word contexts;
generating targeted documents based on an analysis of the initial control set;
reviewing the generated targeted documents based on the machine assisted automatic coding for the initial control set and human-only assisted hard coding of documents;
based on the reviewing of the generated targeted documents, modifying the machine assisted automatic coding and reiterating the generating of the targeted documents and reviewing of the generated targeted documents;
generating a confidence validation by comparing the machine assisted automatic coding to a human-only assisted hard coding of the documents;
based on the confidence validation, determining whether the machine assisted automatic coding is more accurate than the human-only assisted hard coding; and
when the machine assisted automatic coding is not more accurate than the human-only assisted hard coding, evaluating the machine assisted automatic coding with a randomly selected document and modifying the machine assisted automatic coding based on the evaluating.
2 . The method of claim 1 , wherein the document review coding category corresponds to electronic discovery categories comprising responsive, issues, or privilege.
3 . The method of claim 1 , wherein the initial control set is further generated based on the received user input on the documents.
4 . The method of claim 3 , wherein generating the initial control set comprises:
receiving the human-only assisted hard coding of a selected subset of the documents based on the document review coding category; and
analyzing the selected subset of the documents to generate the machine assisted automatic coding.
5 . The method of claim 1 , wherein reviewing the generated targeted documents based on the machine assisted automatic coding for the initial control set and human-only assisted hard coding of the documents comprises:
receiving human assisted validation of the generated targeted documents.
6 . The method of claim 5 , wherein the randomly selected document is a plurality of randomly selected documents used to reseed the initial control set for further evaluation.
7 . The method of claim 1 , wherein when the machine assisted automatic coding is more accurate than the human-only assisted hard coding, generating a selected subset of the documents from the corpus of the documents and corresponding to the document review coding category.
8 . A system for document review in electronic discovery, comprising:
a processor; and
memory storing instructions that, when executed by the processor, causes the system to perform a set of operations, the set of operations comprising:
receiving from user input, a document review coding category;
generating, by machine assisted automatic coding, based on the document review coding category, an initial control set comprising documents from a corpus of documents;
training probabilistic latent semantic analysis (PLSA) on the initial control set to automatically detect concepts within the documents of the corpus via a statistical analysis of word contexts;
performing the machine assisted automatic coding on the documents of the corpus, by executing the PLSA trained on the initial control set to assign coding determinations to the documents of the corpus according to the document review coding category, based on the PLSA's automatic detection of concepts within the documents of the corpus via the statistical analysis of the word contexts;
generating targeted documents based on an analysis of the initial control set;
reviewing the generated targeted documents based on the machine assisted automatic coding for the initial control set and human-only assisted hard coding of documents;
based on the reviewing of the generated targeted documents, modifying the machine assisted automatic coding and reiterating the generating of the targeted documents and reviewing of the generated targeted documents;
generating a confidence validation by comparing the machine assisted automatic coding to the human-only assisted hard coding of the documents;
based on the confidence validation, determining whether the machine assisted automatic coding is more accurate than the human-only assisted hard coding; and
when the machine assisted automatic coding is not more accurate than the human-only assisted hard coding, evaluating the machine assisted automatic coding with a randomly selected document and modifying the machine assisted automatic coding based on the evaluating.
9 . The system of claim 8 , wherein the document review coding category corresponds to electronic discovery categories comprising responsive, issues, or privilege.
10 . The system of claim 8 , wherein the initial control set is further generated based on the received user input on the documents.
11 . The system of claim 10 , wherein generating the initial control set comprises:
receiving the human-only assisted hard coding of a selected subset of the documents based on the document review coding category; and
analyzing the selected subset of the documents to generate the machine assisted automatic coding.
12 . The system of claim 8 , wherein reviewing the generated targeted documents based on the machine assisted automatic coding for the initial control set and human-only assisted hard coding of the documents comprises:
receiving human assisted validation of the generated targeted documents.
13 . The system of claim 12 , wherein the randomly selected document is a plurality of randomly selected documents used to reseed the initial control set for further evaluation.
14 . The system of claim 8 , wherein when the machine assisted automatic coding is more accurate than the human-only assisted hard coding, generating a selected subset of the documents from the corpus of the documents and corresponding to the document review coding category.
15 . A computer product storing instructions that, when executed by a processor, are capable of performing a method for document review in electronic discovery, the method comprising:
receiving from user input, a document review coding category;
generating, by machine assisted automatic coding, based on the document review coding category, an initial control set comprising documents from a corpus of documents;
training probabilistic latent semantic analysis (PLSA) on the initial control set to automatically detect concepts within the documents of the corpus via a statistical analysis of word contexts;
performing the machine assisted automatic coding on the documents of the corpus, by executing the PLSA trained on the initial control set to assign coding determinations to the documents of the corpus according to the document review coding category, based on the PLSA's automatic detection of concepts within the documents of the corpus via the statistical analysis of the word contexts;
generating targeted documents based on an analysis of the initial control set;
reviewing the generated targeted documents based on the machine assisted automatic coding for the initial control set and human-only assisted hard coding of documents;
based on the reviewing of the generated targeted documents, modifying the machine assisted automatic coding and reiterating the generating of the targeted documents and reviewing of the generated targeted documents;
generating a confidence validation by comparing the machine assisted automatic coding to a human-only assisted hard coding of the documents;
based on the confidence validation, determining whether the machine assisted automatic coding is more accurate than the human-only assisted hard coding; and
when the machine assisted automatic coding is not more accurate than the human-only assisted hard coding, evaluating the machine assisted automatic coding with a randomly selected document and modifying the machine assisted automatic coding based on the evaluating.
16 . The computer product of claim 15 , wherein the initial control set is further generated based on the received user input on the documents.
17 . The computer product of claim 16 , wherein generating the initial control set comprises:
receiving the human-only assisted hard coding of a selected subset of the documents based on the document review coding category; and
analyzing the selected subset of the documents to generate the machine assisted automatic coding.
18 . The computer product of claim 15 , wherein reviewing the generated targeted documents based on the machine assisted automatic coding for the initial control set and human-only assisted hard coding of the documents comprises:
receiving human assisted validation of the generated targeted documents.
19 . The computer product of claim 18 , wherein the randomly selected document is a plurality of randomly selected documents used to reseed the initial control set for further evaluation.
20 . The computer product of claim 15 , wherein when the machine assisted automatic coding is more accurate than the human-only assisted hard coding, generating a selected subset of documents from the corpus of documents and corresponding to the document review coding category.