Systems and methods for predictive coding analysis for electronic discovery
Systems and methods for analyzing documents are provided herein. A plurality of documents and user input are received via a computing device. The user input includes hard coding of a subset of the plurality of documents, based on an identified subject or category. Instructions stored in memory are executed by a processor to generate an initial control set, analyze the initial control set to determine at least one seed set parameter, automatically code a first portion of the plurality of documents based on the initial control set and the seed set parameter associated with the identified subject or category, analyze the first portion of the plurality of documents by applying an adaptive identification cycle, and retrieve a second portion of the plurality of documents based on a result of the application of the adaptive identification cycle test on the first portion of the plurality of documents.
1 . A system for analyzing a plurality of documents for electronic discovery using an adaptive identification cycle, comprising:
a processor for training the adaptive identification cycle; and
memory storing instructions that, when executed by the processor, cause the system to perform a set of training operations for training the adaptive identification cycle, the set of training operations comprising:
receiving the plurality of documents via a computing device to be used for training the adaptive identification cycle;
filtering the plurality of documents to produce a subset of the plurality of documents used in training the adaptive identification cycle;
generating an initial control set based on random sampling of the subset of the plurality of documents;
receiving user input corresponding to an identified subject or category;
reviewing the initial control set to determine at least one seed set parameter associated with the identified subject or category;
soft coding a first portion of the plurality of documents, based on the initial control set and the at least one seed set parameter associated with the identified subject or category;
analyzing the first portion of the plurality of documents based on at least one of: the initial control set, user validation of the soft coding of the first portion of the plurality of documents, and a confidence threshold validation, the analyzing the first portion of the plurality of documents comprising:
soft coding a second portion of the plurality of documents with the initial control set;
presenting at least one document from the second portion of the plurality of documents to a human reviewer and allowing the human reviewer to correct at least a portion of the soft coding by performing a hard coding correction, the hard coding correction comprising changing of the at least a portion of the soft coding from the soft coding to a hard coding;
receiving the hard coding correction to the at least a portion of the soft coding of the at least one document;
updating the initial control set with the at least one document having the hard coding correction; and
applying the updated coded control set to additional documents to automatically code the additional documents; and
validating the plurality of documents by determining that the plurality of documents meets the confidence threshold.
2 . The system according to claim 1 , wherein the plurality of documents are partially coded.
3 . The system according to claim 1 , further comprising:
querying for the additional documents that correspond to the hard coding of the plurality of documents using the at least one seed set parameter; and
retrieving the additional documents that correspond to the hard coding of the plurality of documents.
4 . The system according to claim 3 , further comprising generating a confidence score for the additional documents that comprises a computer-generated judgment on soft coding of the additional documents.
5 . The system according to claim 4 , further comprising updating the initial set with the additional documents that correspond to the hard coding of the plurality of documents to create a new seed set.
6 . The system according to claim 5 , further comprising:
identifying non-relevant documents as a set of negative examples;
enriching the set of negative examples to a given total of documents by randomly sampling the non-relevant documents; and
reviewing the non-relevant documents that correspond to the identified subject or category.
7 . The system according to claim 6 , further comprising analyzing the non-relevant documents using a support vector machine that maps the non-relevant documents into an internal high-dimensional representation.
8 . The system according to claim 7 , further comprising computing linear functions on the internal high-dimensional representation to model training examples.
9 . The system according to claim 1 , further comprising retrieving the second portion of the plurality of documents based on a result of application of the adaptive identification cycle on the first portion of the plurality of documents.
10 . The system according to claim 1 , wherein generating the initial set comprises receiving a request for content, the initial set being selected based on the request for content.
11 . A computer product storing instructions that, when executed by a processor, are capable of performing a method of training an adaptive identification cycle for document review in electronic discovery, the method of training an adaptive identification cycle comprising:
receiving a plurality of documents via a computing device to be used for training the adaptive identification cycle;
filtering the plurality of documents to produce a subset of the plurality of documents used in training the adaptive identification cycle;
generating an initial control set based on random sampling of the subset of the plurality of documents;
receiving user input corresponding to an identified subject or category;
reviewing the initial control set to determine at least one seed set parameter associated with the identified subject or category;
soft coding a first portion of the plurality of documents, based on the initial control set and the at least one seed set parameter associated with the identified subject or category;
analyzing the first portion of the plurality of documents based on at least one of: the initial control set, user validation of the soft coding of the first portion of the plurality of documents, and a confidence threshold validation, the analyzing the first portion of the plurality of documents comprising:
soft coding a second portion of the plurality of documents with the initial control set;
presenting at least one document from the second portion of the plurality of documents to a human reviewer and allowing the human reviewer to correct at least a portion of the soft coding by performing a hard coding correction, the hard coding correction comprising changing of the at least a portion of the soft coding from the soft coding to a hard coding;
receiving the hard coding correction to the at least a portion of the soft coding of the at least one document;
updating the initial control set with the at least one document having the hard coding correction; and
applying the updated coded control set to additional documents to automatically code the additional documents; and
validating the plurality of documents by determining that the plurality of documents meets the confidence threshold.
12 . The computer product according to claim 11 , wherein the plurality of documents are partially coded.
13 . The computer product according to claim 11 , further comprising:
querying for the additional documents that correspond to the hard coding of the plurality of documents using the at least one seed set parameter; and
retrieving the additional documents that correspond to the hard coding of the plurality of documents.
14 . The computer product according to claim 13 , further comprising generating a confidence score for the additional documents that comprises a computer-generated judgment on soft coding of the additional documents.
15 . The computer product according to claim 14 , further comprising updating the initial set with the additional documents that correspond to the hard coding of the plurality of documents to create a new seed set.
16 . The computer product according to claim 15 , further comprising:
identifying non-relevant documents as a set of negative examples;
enriching the set of negative examples to a given total of documents by randomly sampling the non-relevant documents; and
reviewing the non-relevant documents that correspond to the identified subject or category.
17 . The computer product according to claim 16 , further comprising analyzing the non-relevant documents using a support vector machine that maps the non-relevant documents into an internal high-dimensional representation.
18 . The computer product according to claim 17 , further comprising computing linear functions on the internal high-dimensional representation to model training examples.
19 . The computer product according to claim 11 , further comprising retrieving the second portion of the plurality of documents based on a result of application of the adaptive identification cycle on the first portion of the plurality of documents.
20 . The computer product according to claim 11 , wherein generating the initial set comprises receiving a request for content, the initial set being selected based on the request for content.
21 . The system according to claim 1 , wherein the confidence threshold is based on a confidence score of the plurality of documents, with the confidence score relating to one or more of relevancy, responsiveness, and privileged nature of the plurality of documents.