System, method, and apparatus for coordinating distributed electronic discovery processing

1. A computer-implemented method for coordinating distributed electronic discovery processing, comprising:

receiving electronic discovery data for each of a plurality of discovery processing sessions, the electronic discovery data comprising a plurality of electronic files, wherein each session is associated with a priority;

determining, with at least one processor, a plurality of discrete tasks for processing the electronic discovery data for each session of the plurality of discovery processing sessions;

determining, with at least one processor, a coordinated distribution of the plurality of discrete tasks to a plurality of available worker machines based at least partially on the priority of each session of the plurality of discovery processing sessions, wherein the plurality of available worker machines comprise a plurality of virtual machines hosted by at least one physical host computer;

assigning, with at least one processor, the plurality of discrete tasks to the plurality of available worker machines based on the coordinated distribution; and

recording, with at least one processor in at least one database, processed data resulting from a performance of each discrete task by a worker machine.

2. The computer-implemented method of claim 1 , wherein recording the processed data results an one original file, further comprising:

detecting, with at least one processor, a duplicate file of the at least one original file in the electronic discovery data based at least partially on hash values of the duplicate file and the original file;

generating, with at least one processor, a new record for the duplicate file;

generating, with at least one processor, a clone of the original file; and

storing the clone in the new record.

3. The computer-implemented method of claim 2 , further comprising generating, with at least one processor, a hash value for each file processed of the plurality of electronic files, wherein the duplicate file is detected by comparing a hash value for the duplicate file with at least one hash value for at least one previously processed file including the original file, such that the hash value for the duplicate file matches at least one hash value for the original file.

4. The computer-implemented method of claim 1 , wherein recording the processed data results in an original file, further comprising:

detecting, with at least one processor, a duplicate file of the original file in the electronic discovery data;

generating, with at least one processor, a new record for the duplicate file; and

storing a pointer to the original file in the new record.

5. The computer-implemented method of claim 4 , further comprising: processing the duplicate file, resulting in extracted data including metadata, wherein storing the pointer in the new record comprises embedding the pointer in the metadata.

6. The computer-implemented method of claim 1 , further comprising:

determining, with at least one processor, a bias for each session of the plurality of discovery sessions, wherein the bias is based at least partially on at least one deadline comprising at least one of the following: a project completion date, a determined deadline, an inputted deadline, or any combination thereof; and

determining, with at least one processor, processing resource allocations for the plurality of discovery processing sessions, wherein the coordinated distribution is determined based at least partially on the determined processing resource allocations such that a session with a higher bias will be assigned more worker machines than a session with a lower bias having the same priority as the session with the higher bias.

7. The computer-implemented method of claim 1 , wherein receiving the electronic discovery data comprises at least one of the following: loading the electronic discovery data from a file system, receiving the electronic discovery data over a network, identifying a location of the electronic discovery data, or any combination thereof.

8. The computer-implemented method of claim 1 , further comprising:

adding at least one worker machine to the plurality of available worker machines;

determining, with at least one processor, a new coordinated distribution of at least a portion of the plurality of discrete tasks to the plurality of available worker machines; and

assigning, with at least one processor, at least a portion of the plurality of discrete tasks to the plurality of available worker machines based on the new coordinated distribution.

9. The computer-implemented method of claim 8 , wherein the new coordinated distribution comprises an allocation of worker machines to active sessions proportional to a current allocation of worker machines to active sessions.

10. The computer-implemented method of claim 1 , further comprising:

generating, with at least one processor, a resource monitor user interface comprising at least one selectable option configured to select a worker machine from the plurality of worker machines;

receiving a user selection of a selected worker machine through the resource monitor user interface; and

initiating a remote desktop connection with the selected worker machine in response to the user selection.

11. The computer-implemented method of claim 1 , wherein only a single discrete task is assigned to an available worker machine at a time.

12. The computer-implemented method of claim 1 , further comprising receiving, from at least one user, a priority and a bias for each discovery processing session of the plurality of discovery processing sessions, wherein determining the coordinated distribution of the plurality of discrete tasks to the plurality of available worker machines comprises:

(a) identifying at least one discovery processing session of the plurality of discovery processing sessions having a highest priority;

(b) identifying a subset of available worker machines of the plurality of available worker machines;

(c) allocating the subset of available worker machines to the at least one discovery processing session having the highest priority based on the bias for each discovery processing session of the at least one discovery processing session; and

(d) repeating steps (a)-(c) for remaining discovery processing sessions of the plurality of discovery processing sessions.

13. A computer program product for coordinating distributed electronic discovery processing for a plurality of discovery processing sessions, comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:

receive electronic discovery data for each of a plurality of discovery processing sessions, the electronic discovery data comprising a plurality of electronic files, wherein each session is associated with a priority;

determine a plurality of discrete tasks for processing the electronic discovery data for each session of the plurality of discovery processing sessions;

determine a coordinated distribution of the plurality of discrete tasks to a plurality of available worker machines based at least partially on the priority of each session of the plurality of discovery processing sessions, wherein the plurality of available worker machines comprise a plurality of virtual machines hosted by at least one physical host computer;

assign the plurality of discrete tasks to the plurality of worker machines based on the coordinated distribution; and

record processed data resulting from a performance of each discrete task by a worker machine.

14. The computer program product of claim 13 , wherein recording the processed data results in an original file, and wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to:

detect a duplicate file of the original file in the electronic discovery data based at least partially on hash values of the duplicate file and the original file;

generate a new record for the duplicate file;

generate a clone of the original file; and

store the clone in the new record.

15. The computer program product of claim 14 , wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to generate a hash value for each file processed of the plurality of electronic files, wherein the duplicate file is detected by comparing a hash value for the duplicate file with at least one hash value for at least one previously processed file including the original file, such that the hash value for the duplicate file matches at least one hash value for the original file.

16. The computer program product of claim 13 , wherein recording the processed data results in an original file, and wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to:

detect a duplicate file of the original file in the electronic discovery data;

generate a new record for the duplicate file; and

store a pointer to the original file in the new record.

17. The computer program product of claim 16 , wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to process the duplicate file, resulting in extracted data including metadata, and wherein storing the pointer in the new record comprises embedding the pointer in the metadata.

18. The computer program product of claim 13 , wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to:

determine a bias for each session of the plurality of discovery sessions based at least partially on at least one deadline; and

determine processing resource allocations for the plurality of discovery processing sessions, wherein the coordinated distribution is determined based at least partially on the determined processing resource allocations such that a session with a higher bias will be assigned more worker machines than a session with a lower bias having the same priority as the session with the higher bias.

19. The computer program product of claim 13 , wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to:

add at least one worker machine to the plurality of available worker machines;

determine a new coordinated distribution of at least a portion of the plurality of discrete tasks to the plurality of available worker machines; and

assign at least a portion of the plurality of discrete tasks to the plurality of available worker machines based on the new coordinated distribution.

20. The computer program product of claim 19 , wherein the new coordinated distribution comprises an allocation of worker machines to active sessions proportional to a current allocation of worker machines to active sessions.

21. The computer program product of claim 13 , wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to:

generate a resource monitor user interface comprising at least one selectable option configured to select a worker machine from the plurality of worker machines;

receive a user selection of a selected worker machine through the resource monitor user interface; and

initiate a remote desktop connection with the selected worker machine in response to the user selection.

22. The computer program product of claim 13 , wherein only a single discrete task is assigned to an available worker machine at a time.

23. The computer program product of claim 13 , wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to receive, from at least one user, a priority and a bias for each discovery processing sessions of the plurality of discovery processing sessions, wherein determining the coordinated distribution of the plurality of discrete tasks to the plurality of available worker machines comprises:

(a) identifying at least one discovery processing session of the plurality of discovery processing sessions having a highest priority;

(b) identifying a subset of available worker machines of the plurality of available worker machines;

(c) allocating the subset of available worker machines to the at least one discovery processing session having the highest priority based on the bias for each discovery processing session of the at least one discovery processing session; and

(d) repeating steps (a)-(c) for remaining discovery processing sessions of the plurality of discovery processing sessions.

24. A system for distributed electronic discovery processing, comprising:

(a) at least one data storage device comprising electronic discovery data;

(b) a plurality of worker machines, wherein each worker machine comprises at least one of a virtual machine and a physical computer programmed or configured to:

(i) receive a discrete processing task to process data from the electronic discovery data;

(ii) perform the discrete processing task, resulting in processed data; and

(iii) record the processed data to at least one data storage device;

(c) an electronic discovery distribution engine executing on at least one computer and in communication with the at least one data storage device and the plurality of worker machines, the electronic discovery distribution engine configured to:

(i) determine a plurality of discrete processing tasks for processing the electronic discovery data; and

(ii) automatically coordinate a distribution of the plurality of discrete processing tasks to available worker machines of the plurality of worker machines.

25. The system of claim 24 , wherein the electronic discovery distribution engine comprises a loading module and a master module, wherein the loading module is programmed or configured to receive user input comprising at least one of the following: an identification of a data source for the electronic discovery data, an instruction to stop or start at least one discovery processing session of the plurality of discovery processing sessions, user preferences, or any combination thereof, and wherein the master module is programmed or configured to determine the plurality of discrete processing tasks for processing the electronic discovery data and automatically coordinate the distribution of the plurality of discrete processing tasks to the available worker machines.

26. The system of claim 25 , wherein recording the processed data to the at least one data storage device results in an original file, and wherein the electronic discovery distribution engine is further configured to:

detect a duplicate file of the original file in the electronic discovery data based at least partially on hash values of the duplicate file and the original file;

generate a new record for the duplicate file;

generate a clone of the original file; and

store the clone in the new record.

27. The system of claim 26 , wherein the electronic discovery distribution engine is further configured to cause the at least one processor to generate a hash value for each file processed of the plurality of electronic files, wherein the duplicate file is detected by comparing a hash value for the duplicate file with at least one hash value for at least one previously processed file including the original file, such that the hash value for the duplicate file matches at least one hash value for the at least one original file.

28. The system of claim 25 , wherein recording the processed data to the at least one data storage device results in at least one original file, and wherein the electronic discovery distribution engine is further configured to:

detect a duplicate file of the original file in the electronic discovery data;

generate a new record for the duplicate file; and

store a pointer to the original file in the new record.

29. The system of claim 28 , wherein the electronic discovery distribution engine is further configured to cause the at least one processor to process the duplicate file, resulting in extracted data including metadata, and wherein storing the pointer in the new record comprises embedding the pointer in the metadata.

30. The system of claim 24 , wherein the electronic discovery distribution engine is further configured to:

determine a bias for each session of the plurality of discovery sessions based at least partially on at least one deadline; and

determine processing resource allocations for the plurality of discovery processing sessions, wherein the coordinated distribution is determined based at least partially on the determined processing resource allocations such that a session with a higher bias will be assigned more worker machines than a session with a lower bias having the same priority as the session with the higher bias.

31. The system of claim 24 , wherein the electronic discovery distribution engine is further configured to:

add at least one worker machine to the plurality of available worker machines;

determine a new coordinated distribution of at least a portion of the plurality of discrete tasks to the plurality of available worker machines; and

assign at least a portion of the plurality of discrete tasks to the plurality of available worker machines based on the new coordinated distribution.

32. The system of claim 24 , wherein the electronic discovery distribution engine is further configured to receive, from at least one user, a priority and a bias for each discovery processing sessions of the plurality of discovery processing sessions, wherein determining the coordinated distribution of the plurality of discrete tasks to the plurality of available worker machines comprises:

(a) identifying at least one discovery processing session of the plurality of discovery processing sessions having a highest priority;

(b) identifying a subset of available worker machines of the plurality of available worker machines;

(c) allocating the subset of available worker machines to the at least one discovery processing session having the highest priority based on the bias for each discovery processing session of the at least one discovery processing session; and

(d) repeating steps (a)-(c) for remaining discovery processing sessions of the plurality of discovery processing sessions.