Systems, methods, and apparatus for hierarchical database
This disclosure describes systems, methods, and apparatus for creating or updating a table in a fully-indexed hierarchical database for storing both structured and unstructured data, where the database structure rather than separate index files provides indexing. The database includes a hierarchy of nodes where the lowest nodes represent raw data and hard links to the raw data. Data records in a table, write request, update request, etc. can be exploded into attributes and values for those attributes, and then each value is stored in a separate file at the lowest nodes of the hierarchy.
1. A fully-indexed hierarchical database system for storing both structured and unstructured data, where the database structure rather than separate index files provides indexing, the database including:
an HTTP/TCP server configured to receive a request from a client to write data records to a hierarchical database representing a table having rows and attributes; and
a non-transitory, tangible computer readable storage medium, encoded with processor readable instructions for creating or updating a table in the fully-indexed hierarchical database, the method comprising:
exploding the data records into attributes and values, where each of the values corresponds to one of the attributes and one of the records;
writing raw data nodes under a table node, the raw data nodes comprising:
a raw data file for each of the values, the raw data files being children of raw data attribute nodes, the raw data attribute nodes being children of a raw data node, the raw data node being a child of the table node; and
writing hard link nodes under the table node, the hard link nodes comprising:
a hard link file for each of the values, the hard link files containing hard links to corresponding ones of the raw data files, the hard link files being children of value nodes corresponding to the values, the value nodes being children of hard link attribute nodes, the hard link attribute nodes being children of the table node.
2. The fully-indexed hierarchical database system of claim 1 , wherein a compiled version of the database is less than 100 MB in storage size.
3. The fully-indexed hierarchical database system of claim 2 , wherein the compiled version of the database is less than 50 MB in storage size.
4. The fully-indexed hierarchical database system of claim 1 , wherein the fully-indexed hierarchical database has a dynamic schema.
5. A method of creating or updating a table in a fully-indexed hierarchical database for storing both structured and unstructured data, where the database structure rather than separate index files provides indexing, the method comprising:
receiving a request to write data records to a hierarchical database representing a table having rows and attributes;
exploding the data records into attributes and values, where each of the values corresponds to one of the attributes and one of the records;
writing raw data nodes under a table node, the raw data nodes comprising:
a raw data file for each of the values, the raw data files being children of raw data attribute nodes, the raw data attribute nodes being children of a raw data node, the raw data node being a child of the table node; and
writing hard link nodes under the table node, the hard link nodes comprising:
a hard link file for each of the values, the hard link files containing hard links to corresponding ones of the raw data files, the hard link files being children of value nodes corresponding to the values, the value nodes being children of hard link attribute nodes, the hard link attribute nodes being children of the table node.
6. The method of claim 5 , wherein the request is made via one of the following data formats: SQL, NoSQL, JSON, XML, CSV via HTTP REST API and/or command line interface.
7. The method of claim 5 , wherein one of the raw data attribute nodes and one of the hard link attribute nodes corresponds to unique identifiers of the table.
8. The method of claim 5 , wherein the fully-indexed hierarchical database has a dynamic schema.
9. The method of claim 5 , wherein a compiled version of the database is less than 100 MB in storage size.
10. The method of claim 9 , wherein the compiled version of the database is less than 50 MB in storage size.
11. A non-transitory, tangible computer readable storage medium, encoded with processor readable instructions to perform a method for creating or updating a table in a fully-indexed hierarchical database for storing both structured and unstructured data, the method comprising:
receiving a request to write data records to a hierarchical database representing a table having rows and attributes;
exploding the data records into attributes and values, where each of the values corresponds to one of the attributes and one of the records;
writing raw data nodes under a table node, the raw data nodes comprising:
a raw data file for each of the values, the raw data files being children of raw data attribute nodes, the raw data attribute nodes being children of a raw data node, the raw data node being a child of the table node; and
writing hard link nodes under the table node, the hard link nodes comprising:
a hard link file for each of the values, the hard link files containing hard links to corresponding ones of the raw data files, the hard link files being children of value nodes corresponding to the values, the value nodes being children of hard link attribute nodes, the hard link attribute nodes being children of the table node.
12. The non-transitory, tangible computer readable storage medium of claim 11 , wherein the request is made via one of the following data formats: SQL, NoSQL, JSON, XML, CSV via HTTP REST API and/or command line interface.
13. The non-transitory, tangible computer readable storage medium of claim 11 , wherein one of the raw data attribute nodes and one of the hard link attribute nodes corresponds to unique identifiers of the table.
14. The non-transitory, tangible computer readable storage medium of claim 11 , wherein the fully-indexed hierarchical database has a dynamic schema.
15. The non-transitory, tangible computer readable storage medium of claim 11 , wherein a compiled version of the database is less than 100 MB in storage size.
16. The non-transitory, tangible computer readable storage medium of claim 15 , wherein the compiled version of the database is less than 50 MB in storage size.