Systems and methods for parallelized custom data-processing and search
This invention provides systems and methods for data processing by means of an ongoing background process on an end-user's computer. As a user receives and generates data, files are analyzed. A container file is opened into the volatile memory and its contents (including data and metadata) are extracted, without requiring an index to be created. The extracted components are analyzed based on predefined characteristics.
1. A method for processing data, the method comprising:
providing a system comprising one or more processors coupled to memory containing instructions executable by the one or more processors to cause the system to perform, in response to receiving initial input from a user, the following steps:
opening an archive container file into a volatile memory;
extracting, using a first process executed on a first processor, a first computer file from the archive container file without indexing the entire archive container file and without creating an index;
determining, using the first process, that the first computer file has a second computer file stored therein;
causing, using the first process, a second process to extract the second computer file;
extracting, using the second process executed on a second processor operating in parallel with the first processor, the second computer file;
determining, using the second process, that the second computer file satisfies a criterion defined by the user; and
performing an operation, defined by the user, on the second computer file;
wherein the system is configured to run, via the one or more processors, each of the first and second processes as ongoing processes and in the background so as to continually monitor a target computer to automatically detect and identify, in real time, evidence of security-related issues or data points of interests associated with security-related issues.
2. The method of claim 1 , wherein the volatile memory is a random-access memory.
3. The method of claim 1 , wherein the computer files comprise content and metadata.
4. The method of claim 1 , wherein the criterion is selected from an example group consisting of: 1) containing a third computer file therein; 2) being a particular file format; 3) containing certain data; 4) containing certain metadata; 5) containing a URL known to be associated with phishing or malware; 6) containing certain keywords; 7) containing certain dates; and 8) being written by a certain author.
5. The method of claim 1 , wherein the operation is selected from a group consisting of: 1) applying a heuristic rule to the file; 2) analyzing content of the extracted file; 3) searching the file; 4) saving the file to a non-volatile memory; 5) moving the file to a different location; 6) extracting certain data from the file; 7) extracting certain metadata from the file; 8) adding the file to a list; 9) sending an alert; and 10) deleting the file.
6. A system for processing data, the system comprising:
at least two processors coupled to a volatile memory and configured to operate in parallel in response to receiving initial input from a user,
wherein a first processor is configured to:
open an archive container file into a volatile memory;
extract, using a first process executed on a first processor, a first computer file from the archive container file without indexing the entire archive container file and without creating an index;
cause, using the first process, a second process to extract the second computer file;
wherein a second processor is configured to:
extract, using the second process executed on the second processor, the second computer file;
determine, using the second process, that the second computer file satisfies a criterion defined by the user; and
perform an operation, defined by the user, on the second computer file,
wherein the system is configured to run, via the first and second processors, each of the first and second processes as ongoing processes and in the background so as to continually monitor a target computer to automatically detect and identify, in real time, evidence of security-related issues or data points of interests associated with security-related issues.
7. The system of claim 6 , wherein the volatile memory is a random-access memory.
8. The system of claim 6 , wherein the computer files comprise content and metadata.
9. The system of claim 6 , wherein the criterion is selected from a group consisting of: a) containing a third computer file therein; b) being a particular file format; c) containing certain data; d) containing certain metadata; e) containing a URL known to be associated with phishing or malware; f) containing certain keywords; g) containing certain dates; and h) being written by a certain author.
10. The system of claim 6 , wherein the operation is selected from a group consisting of: a) applying a heuristic rule to the file; b) analyzing content of the extracted file; c) searching the file; d) saving the file to a non-volatile memory; e) moving the file to a different location; f) extracting certain data from the file; g) extracting certain metadata from the file; h) adding the file to a list; i) sending an alert; and j) deleting the file.