System and method for generating, maintaining, and querying a database for computer investigations cross-reference to related application
A computer investigation system and method organizes information in a hive. Information in a target device is parsed by a processor, and facts and links are extracted from the data. The processor identifies the fact type for each identified fact, and further generates a fact ID for the fact. The information for the fact is stored in the hive in a fact table associated with the fact type. The processor also identifies the link type for each identified link, and further generates a link ID for the link. The information for the link ID is stored in the hive in a link table associated with the identified link type. A query language that is adapted to work with the hive allows querying of data stored in the hive.
1. A method for querying a database comprising tables stored in a hardware data storage device, the method comprising:
receiving a search query describing a filter, wherein the filter includes a plurality of conditions, wherein each of the conditions is enclosed in a gate ending with a particular punctuation mark;
processing, by a hardware processor, the search query, wherein in processing the search query, the processor:
accesses the tables stored in the hardware data storage device, wherein the tables store each piece of data as either a unique fact or a unique link, wherein all facts are stored at a same level and all facts are connected to other facts via one or more links;
retrieves data from the tables stored in the hardware data storage device; and
searches the data retrieved from the tables stored in the hardware data storage device for identifying a match for each of the conditions; and
returning, by the hardware processor, search results in response to identifying the match for each of the conditions in the data retrieved from the tables stored in the hardware data storage device.
2. The method of claim 1 , wherein a first one of the tables includes entries for facts of a first type, wherein a second one of the tables includes entries for facts of a second type, wherein each entry includes a unique identifier and metadata for the fact.
3. The method of claim 2 , wherein the unique identifier for a particular fact is based on the metadata of the particular fact.
4. The method of claim 2 , wherein a third one of the tables includes entries for links associating a first fact in the first table with a second fact in the second table.
5. The method of claim 1 , wherein each of the tables stored in the data storage device comprises either a fact table or a link table.
6. The method of claim 5 , wherein for each of the fact tables, each entry includes a unique identifier and metadata for the fact contained in the entry.
7. The method of claim 5 , wherein for each of the link tables, each entry includes a link associating a fact in one of the fact tables with a fact in another of the fact tables.
8. The method of claim 1 , further comprising retrieving, by an examining machine, the facts in the fact tables from one or more target machines and storing the facts in the entries of the fact tables in the data storage device.
9. The method of claim 1 , wherein the unique identifier for a particular fact is globally unique and is independent of the metadata of the particular fact.
10. The method of claim 1 , wherein the filter includes a boolean match constraint on the plurality of conditions.
11. The method of claim 10 , wherein the boolean match constraint is selected from the group including: match all; match any; match none; match a specified number.
12. A computer investigation system comprising:
an examining machine having a hardware processor and an interface configured to receive a search query, the search query describing a filter, wherein the filter includes a plurality of conditions, wherein each of the conditions is enclosed in a gate ending with a particular punctuation mark; and
a hardware data storage device communicatively coupled to the examining machine, the hardware data storage device storing a database having a plurality of tables;
wherein the hardware processor is configured to execute the search query, and thereby access the tables stored in the hardware data storage device, wherein the tables store each piece of data as either a unique fact or a unique link, wherein all facts are stored at a same level and all facts are connected to other facts via one or more links, retrieve data from the tables stored in the hardware data storage device, search the data retrieved from the tables stored in the hardware data storage device, and identify a match for each of the conditions; and
wherein the hardware processor is further configured to return search results in response to identifying the match for each of the conditions in the data retrieved from the tables stored in the hardware data storage device.
13. The computer investigation system of claim 12 , wherein a first one of the tables includes entries for facts of a first type, wherein a second one of the tables includes entries for facts of a second type, wherein each entry includes a unique identifier and metadata for the fact.
14. The computer investigation system of claim 13 , wherein a third one of the tables includes entries for links associating a first fact in the first table with a second fact in the second table.
15. The computer investigation system of claim 12 , wherein each of the tables stored in the data storage device comprises either a fact table or a link table.
16. The computer investigation system of claim 15 , wherein for each of the fact tables, each entry includes a unique fact identifier and metadata for the fact contained in the entry, and wherein for each of the link tables, each entry includes a link associating a fact in one of the fact tables with a fact in another of the fact tables, and wherein each entry includes a unique link identifier.
17. The computer investigation system of claim 12 , further comprising retrieving, by an examining machine, the facts in the fact tables from one or more target machines and storing the facts in the entries of the fact tables in the data storage device.
18. The computer investigation system of claim 12 , wherein the unique identifier for a particular fact is globally unique and is independent of the metadata of the particular fact.
19. The computer investigation system of claim 12 , wherein the filter includes a boolean match constraint on the plurality of conditions.
20. A computer program product for querying a database comprising tables stored in a hardware data storage device, the computer program product comprising a non-transitory computer-readable medium storing instructions executable by one or more hardware processors to cause the hardware processors to perform:
receiving a search query describing a filter, wherein the filter includes a plurality of conditions, wherein each of the conditions is enclosed in a gate ending with a particular punctuation mark;
processing, by the hardware processors, the search query, wherein in processing the search query, the hardware processors access the tables stored in the hardware data storage device, wherein the tables store each piece of data as either a unique fact or a unique link, wherein all facts are stored at a same level and all facts are connected to other facts via one or more links, retrieve data from the tables stored in the hardware data storage device, and search the data retrieved from the tables stored in a hardware data storage device for identifying a match for each of the conditions; and
returning, by the hardware processors, search results in response to identifying the match for each of the conditions in the data retrieved from the tables stored in the hardware data storage device.