Graph database storage engine
A graph database storage engine is disclosed. In various embodiments, data associated with a node comprising a graph is received via a communication interface. The graph includes a plurality of nodes and a plurality of relationships, and wherein each node in the plurality of nodes has associated therewith a corresponding set of node data and a corresponding set of relationship data. The received data associated with the node is stored in a data storage device in a data record associated with the node, wherein the data record is stored in a static block of a fixed size stored on a single page of data on the data storage device and the data record includes at least a subset of the corresponding set of node data and the corresponding set of relationship data of the node.
1 . A graph database system, comprising:
a communication interface; and
a processor coupled to the communication interface and configured to:
receive, via the communication interface, data associated with a node comprising a graph, wherein the graph includes a plurality of nodes and a plurality of relationships, and wherein each node in the plurality of nodes has associated therewith a corresponding set of node data and a corresponding set of relationship data; and
store the received data associated with the node in a data storage device in a data record associated with the node, wherein the data record is stored in a static block of a fixed size stored on a single page of data on the data storage device and the data record includes at least a subset of the corresponding set of node data and the corresponding set of relationship data of the node;
wherein the single page of data on the data storage device comprises a plurality of static blocks of the fixed size each of which is associated with a corresponding one of the plurality of nodes, a first subset of the static blocks comprising a pair of fixed sized records including a node record comprising node data of the node and a relationship record comprising relationship data of the node and a second subset of the static blocks comprising additional node data of associated with a node with which a corresponding static block in the first subset is associated and to which the corresponding static block in the first subset includes a pointer.
2 . The system of claim 1 , wherein the corresponding set of node data of a node includes a node label.
3 . The system of claim 1 , wherein the corresponding set of node data of a node includes one or more properties of the node.
4 . The system of claim 1 , wherein the processor is configured to store in the data storage device, for each node included in the graph, one or more data records associated uniquely with that node, the data records being stored in a corresponding static block of the fixed size stored on a corresponding single page of data on the data storage device, each such data record including at least a subset of a set of node data and a set of relationship data of corresponding to that node.
5 . The system of claim 1 , wherein the data record comprises a first data record and the processor is configured to store in the same static block as the first data record a second data record.
6 . The system of claim 5 , wherein the first data record and the second data record are of the same size.
7 . The system of claim 5 , wherein the first data record stores node data, including one or more of a node label and a node property.
8 . The system of claim 7 , wherein the second data record stores node relationship data.
9 . The system of claim 1 , wherein the corresponding set of node data includes data that exceeds the size of the data record and the data record includes a pointer to a dynamic data store configured to store at least a portion of the corresponding set of node data that is not stored in the data record.
10 . The system of claim 9 , wherein the dynamic data store is stored in the data storage device on a different page than the page on which the data record is stored.
11 . The system of claim 9 , wherein the dynamic data store includes a set of one or more dynamic data records of a same fixed size.
12 . The system of claim 11 , wherein the processor is configured to grow or shrink the dynamic data store in units of the same fixed size as needed to store node data that does not fit in the data record.
13 . The system of claim 1 , wherein one or both of the corresponding set of node data and the corresponding set of relationship data of the node comprises property data and the processor is configured to store the property data as a set of key value pairs.
14 . The system of claim 13 , wherein the keys and corresponding value sizes are stored separately from the values.
15 . The system of claim 13 , wherein the keys and values are sorted by value size to facilitate fast traversal.
16 . The system of claim 1 , wherein the relationship data is stored by type for fast filtering.
17 . The system of claim 16 , wherein within each type the relationship data is stored with outgoing only relationships first, then bidirectional or loop relationships second, then incoming relationships third to facilitate fast filtering.
18 . The system of claim 1 , further comprising a memory and wherein the processor is further configured to retrieve from the data storage and store in the memory said single page of data that includes the data record associated with the node.
19 . A method of storing graph data, comprising:
receiving, via a communication interface, data associated with a node comprising a graph, wherein the graph includes a plurality of nodes and a plurality of relationships, and wherein each node in the plurality of nodes has associated therewith a corresponding set of node data and a corresponding set of relationship data; and
storing the received data associated with the node in a data storage device in a data record associated with the node, wherein the data record is stored in a static block of a fixed size stored on a single page of data on the data storage device and the data record includes at least a subset of the corresponding set of node data and the corresponding set of relationship data of the node;
wherein the single page of data on the data storage device comprises a plurality of static blocks of the fixed size each of which is associated with a corresponding one of the plurality of nodes, a first subset of the static blocks comprising a pair of fixed sized records including a node record comprising node data of the node and a relationship record comprising relationship data of the node and a second subset of the static blocks comprising additional node data of associated with a node with which a corresponding static block in the first subset is associated and to which the corresponding static block in the first subset includes a pointer.
20 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving, via a communication interface, data associated with a node comprising a graph, wherein the graph includes a plurality of nodes and a plurality of relationships, and wherein each node in the plurality of nodes has associated therewith a corresponding set of node data and a corresponding set of relationship data; and
storing the received data associated with the node in a data storage device in a data record associated with the node, wherein the data record is stored in a static block of a fixed size stored on a single page of data on the data storage device and the data record includes at least a subset of the corresponding set of node data and the corresponding set of relationship data of the node;
wherein the single page of data on the data storage device comprises a plurality of static blocks of the fixed size each of which is associated with a corresponding one of the plurality of nodes, a first subset of the static blocks comprising a pair of fixed sized records including a node record comprising node data of the node and a relationship record comprising relationship data of the node and a second subset of the static blocks comprising additional node data of associated with a node with which a corresponding static block in the first subset is associated and to which the corresponding static block in the first subset includes a pointer.