Systems and methods for parallelized custom data-processing and search
This invention provides systems and methods for data processing by means of an ongoing background process on an end-user's computer. As a user receives and generates data, files are analyzed. A container file is opened into the volatile memory and its contents (including data and metadata) are extracted, without requiring an index to be created. The extracted components are analyzed based on predefined characteristics.
1. A computer-implemented method for processing data generated by a user on a user's computer, the method comprising:
providing a control computer having an application program interface configured to communicate with the user's computer;
receiving, at the control computer via the application program interface, a first computer file generated by the user on the user's computer, as the user generates said first computer file;
determining, using a first process, that the first computer file has a second computer file stored therein;
applying a heuristic rule to determine a subsequent action to be applied to the first computer file, the subsequent action selected from:
(1) applying a first user-defined operation on the first computer file without first extracting the second computer file from the first computer file; and
(2) causing a second process to extract the second computer file from the first computer file, said second computer file then being an extracted second computer file, and performing a second user-defined operation on the extracted second computer file; and
performing the subsequent action.
2. The method of claim 1 , wherein the first computer file has a first size, and the heuristic rule comprises a heuristic rule is based on the first size of the first computer file.
3. The method of claim 1 , wherein the second computer file has a second size, and the heuristic is based on the second size of the second computer file.
4. The method of claim 1 , wherein the heuristic rule is based on user-defined parameters.
5. The method of claim 1 , further comprising, prior to performing the second user-defined operation on the extracted second computer file, determining that the second computer file meets a user-defined criterion.
6. The computer-implemented method of claim 5 , wherein said second user-defined operation comprises indexing, in real time, each such second computer file that satisfies the user-defined criterion.
7. The computer-implemented method of claim 6 , wherein:
the user-defined criterion correlates the second computer file to an eDiscovery search; and wherein:
said second user-defined operation comprises producing an index of documents responsive to the eDiscovery search.
8. The computer-implemented method of claim 1 , wherein said second user-defined operation comprises saving the second computer file to a non-volatile memory.
9. The computer-implemented method of claim 1 , wherein receiving, at the control computer, said computer file generated by the user on the user's computer comprises receiving said computer file in a volatile memory.
10. The computer-implemented method of claim 1 , further comprising continually monitoring the user's computer to automatically index, in real time, each computer file generated by the user on the user's computer.
11. A computer-implemented system for processing data generated by a user on a user's computer, the system comprising:
a control computer remote from the user's computer, the control computer having an application program interface configured to communicate with the user's computer, the control computer configured to:
receive, via the application program interface, a first computer file generated by the user on the user's computer, as the user generates said first computer file;
determine, using a first process, that the first computer file has a second computer file stored therein;
apply a heuristic rule to determine a subsequent action to be applied to the first computer file, the subsequent action selected from:
(1) applying a first user-defined operation on the first computer file without first extracting the second computer file from the first computer file; and
(2) causing a second process to extract the second computer file from the first computer file, said second computer file then being an extracted second computer file, and performing a second user-defined operation on the extracted second computer file; and to
perform the subsequent action.
12. The computer-implemented system of claim 11 , wherein the control computer is further configured to, prior to performing the second user-defined operation on the extracted second computer file, determine that the second computer file meets a user-defined criterion.
13. The computer-implemented system of claim 12 ,
the user-defined criterion correlates the second computer file to an eDiscovery search; and wherein:
said second user-defined operation comprises producing an index of documents responsive to the eDiscovery search.
14. The computer-implemented system of claim 11 , wherein the first computer file has a first size, and the heuristic rule is based on the first size of the first computer file.
15. The computer-implemented system of claim 11 , wherein the control computer is further configured to:
continually monitor the user's computer to automatically index, in real time, each computer file generated by the user on the user's computer.
16. A non-transitory memory device having data stored thereon, the data comprising instructions which, when executed by a processor of a control computer, cause the processor to perform a process, the process comprising:
providing an application program interface configured to communicate with a user's computer;
receiving, at the control computer via the application program interface, a first computer file generated on the user's computer, as the said computer file is generated;
determining, using a first process, that the first computer file has a second computer file stored therein;
applying a heuristic rule to determine a subsequent action to be applied to the first computer file, the subsequent action selected from:
(1) applying a first user-defined operation on the first computer file without first extracting the second computer file from the first computer file; and
(2) causing a second process to extract the second computer file from the first computer file, said second computer file then being an extracted second computer file, and performing a second user-defined operation on the extracted second computer file; and
performing the subsequent action.
17. The non-transitory memory device of claim 16 , wherein the first computer file has a first size, and the heuristic rule is based on the first size of the first computer file.
18. The non-transitory memory device of claim 16 , wherein the second computer file has a second size, and the heuristic rule is based on the second size of the second computer file.
19. The non-transitory memory device of claim 16 , wherein the process further comprises:
prior to performing the second user-defined operation on the extracted second computer file, determining that the second computer file meets a user-defined criterion.
20. The non-transitory memory device of claim 19 , wherein said second user-defined operation comprises indexing, in real time, each such second computer file that satisfies the user-defined criterion.