Efficient synchronization mechanisms for graph databases
Implementations for I/O efficient synchronization on a graph database are provided. One example includes a computing system comprising: processing circuitry and memory storing instructions that, during execution, causes the processing circuitry to: implement the graph database using at least: a read-write node and a read-only node operating in cache memory; and a shared storage for data persistence, wherein the shared storage stores a mapping table of the graph database; receive an update query; update the read-write node based on the update query; write, using the read-write node, a set of logs corresponding to the update query to a write-ahead log journal in the shared storage; and retrieve, using the read-only node, the set of logs from the write-ahead log journal.
1 . A computing system for efficient synchronization on a graph database, the computing system comprising:
processing circuitry and memory storing instructions that, during execution, causes the processing circuitry to:
implement the graph database using at least:
a read-write node and a read-only node operating in cache memory; and
a shared storage for data persistence, wherein the shared storage stores a mapping table of the graph database;
receive an update query;
update the read-write node based on the update query;
write, using the read-write node, a set of logs corresponding to the update query to a write-ahead log (WAL) journal in the shared storage;
retrieve, using the read-only node, the set of logs corresponding to the update query from the WAL journal;
asynchronously flush the cache memory of the read-write node to the shared storage; and
after the flushing of the cache memory of the read-write node to the shared storage, update the mapping table of the graph database in the shared storage based on the update query.
2 . The computing system of claim 1 , wherein the instructions, during execution, further causes the processing circuitry to:
synchronous with the updating of the mapping table in the shared storage, write a log in the WAL journal indicating the shared storage has been updated; and
retrieve, using the read-only node, the log indicating the shared storage has been updated.
3 . The computing system of claim 2 , wherein the instructions, during execution, further causes the processing circuitry to:
update the read-only node using the retrieved set of logs corresponding to the update query; and
discard logs up to the log indicating the shared storage has been updated, including the retrieved set of logs corresponding to the update query.
4 . The computing system of claim 1 , wherein the instructions, during execution, further causes the processing circuitry to:
receive, using the read-only node, a query requesting access to a data element affected by the update of the read-write node;
update the read-only node using the retrieved set of logs corresponding to the update query; and
return, using the read-only node, a query result containing the data element.
5 . The computing system of claim 4 , wherein the instructions, during execution, further causes the processing circuitry to:
synchronous with the updating of the mapping table in the shared storage, write a log in the WAL journal indicating the shared storage has been updated;
retrieve, using the read-only node, the log indicating the shared storage has been updated; and
discard logs up to the log indicating the shared storage has been updated, including the retrieved set of logs corresponding to the update query.
6 . The computing system of claim 4 , wherein updating the read-only node using the retrieved set of logs corresponding to the update query comprises:
performing actions described in the retrieved set of logs corresponding to the update query to update a mapping of the graph database in the read-only node; and
retrieving, using the read-only node, one or more pages from the shared storage using the mapping table of the graph database in the shared storage.
7 . The computing system of claim 6 , wherein updating the read-only node using the retrieved set of logs corresponding to the update query further comprises:
if the shared storage does not contain a page described in the retrieved set of logs corresponding to the update query, creating a copy of the page in the cache memory of the read-only node.
8 . The computing system of claim 1 , wherein updating the read-write node based on the update query comprises performing a page split.
9 . The computing system of claim 1 , wherein the shared storage is an append-only shared storage.
10 . The computing system of claim 1 , wherein the graph database comprises a Bw-tree graph database.
11 . A method for efficiency synchronization on a graph database, the method comprising:
implementing the graph database using at least:
a read-write node and a read-only node operating in cache memory; and
a shared storage for data persistence, wherein the shared storage stores a mapping table of the graph database;
receiving an update query;
updating the read-write node based on the update query;
writing, using the read-write node, a set of logs corresponding to the update query to a write-ahead log (WAL) journal in the shared storage;
retrieving, using the read-only node, the set of logs corresponding to the update query from the WAL journal;
asynchronously flushing the cache memory of the read-write node to the shared storage; and
after the flushing of the cache memory of the read-write node to the shared storage, updating the mapping table of the graph database in the shared storage based on the update query.
12 . The method of claim 11 , further comprising:
synchronous with the updating of the mapping table in the shared storage, writing a log in the WAL journal indicating the shared storage has been updated; and
retrieving, using the read-only node, the log indicating the shared storage has been updated.
13 . The method of claim 12 , further comprising:
updating the read-only node using the retrieved set of logs corresponding to the update query; and
discarding logs up to the log indicating the shared storage has been updated, including the retrieved set of logs corresponding to the update query.
14 . The method of claim 11 , further comprising:
receiving, using the read-only node, a query requesting access to a data element affected by the update of the read-write node;
updating the read-only node using the retrieved set of logs corresponding to the update query; and
returning, using the read-only node, a query result containing the data element.
15 . The method of claim 14 , further comprising:
synchronous with the updating of the mapping table in the shared storage, writing a log in the WAL journal indicating the shared storage has been updated;
retrieving, using the read-only node, the log indicating the shared storage has been updated; and
discarding logs up to the log indicating the shared storage has been updated, including the retrieved set of logs corresponding to the update query.
16 . The method of claim 14 , wherein updating the read-only node using the retrieved set of logs corresponding to the update query comprises:
performing actions described in the retrieved set of logs corresponding to the update query to update a mapping of the graph database in the read-only node;
retrieving, using the read-only node, one or more pages from the shared storage using the mapping table of the graph database in the shared storage; and
if the shared storage does not contain a page described in the retrieved set of logs corresponding to the update query, creating a copy of the page in the cache memory of the read-only node.
17 . The method of claim 11 , wherein updating the read-write node based on the update query comprises performing a page split.
18 . The method of claim 11 , wherein the shared storage is an append-only shared storage.
19 . The method of claim 11 , wherein the graph database comprises a Bw-tree graph database.
20 . A method for efficiency synchronization on a Bw-tree graph database, the method comprising:
implementing the Bw-tree graph database using at least:
a read-write node and a read-only node operating in cache memory; and
an append-only shared cloud storage for data persistence;
updating the read-write node based on a received update query;
writing, using the read-write node, a set of logs corresponding to the received update query to a write-ahead log (WAL) journal in the append-only shared cloud storage;
retrieving, using the read-only node, the set of logs corresponding to the update query from the WAL journal asynchronously flushing the cache memory of the read-write node to the append-only shared cloud storage;
updating the append-only shared cloud storage based on the update query;
synchronous with the updating of the append-only shared cloud storage, writing a log in the WAL journal indicating the append-only shared cloud storage has been updated; and
synchronizing the read-only node to be consistent with the read-write node.