Using persistent memory and remote direct memory access to reduce write latency for database logging
Described is an approach to implement persistent memory to hold log records upon a commit for a database system. The approach may use RDMA techniques to hold change records at a remote persistent memory location.
1 . A method, comprising:
executing operations for a first transaction in a first database at a database node of multiple database nodes, wherein
the database node of the multiple database nodes comprises the first database,
the multiple database nodes share use of a persistent memory within a remote storage appliance to hold log records for the first transaction, and
the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective commits of respective transactions executing on respective database nodes;
generating log data for the operations for the first transaction at the first database;
initiating a commit of the first transaction;
sending the log data together with a request to write the log data to the persistent memory at the remote storage appliance; and
committing the first transaction after successfully sending the request, by the database node, together with the log data for the first transaction using a one-sided protocol to the persistent memory, without the database node waiting for acknowledgment from the remote storage appliance.
2 . The method of claim 1 , further comprising:
executing a separate, second transaction that implements a separate uncommitted change to a second database at a second database node of the multiple database nodes, wherein
the second transaction implements an uncommitted change to the first database instance at the database node of the multiple database nodes, and
the first and second transactions are respectively executed on the first and the second database nodes at least by sharing the persistent memory in the remote storage appliance for respective commits of respective first and second transactions.
3 . The method of claim 1 , wherein
the commit is initiated after the first transaction implements an uncommitted change to the first database on the database node,
the remote storage appliance comprises the persistent memory and the persistent storage that includes one or more storage devices, and
a remote direct memory access (RDMA) operation is used to store at least the log data to the persistent memory.
4 . The method of claim 3 , wherein the log data is stored at the persistent memory with the one-sided RDMA protocol that comprises a one-sided RDMA protocol where no response is required from the remote storage appliance.
5 . The method of claim 1 , wherein a zero-copy operation is implemented to store the log data at the persistent memory while bypassing a CPU at the remote storage appliance.
6 . The method of claim 1 , further comprising:
after moving the log data for the first transaction to the persistent storage within the remote storage appliance, removing the log data for the first transaction from the persistent memory, wherein the log data is stored at the persistent memory and committed upon a successful remote direct memory access (RDMA) send operation by the database node.
7 . The method of claim 1 , wherein
the first transaction is committed prior to the log data being stored in a persistent storage device in the remote storage appliance,
the request is a one-sided request sent via the one-sided protocol,
the first database sends the request without waiting for the acknowledgement from the remote storage appliance, and
the database node of the multiple database nodes includes one or more database processes to operate over the first database at the database node.
8 . The method of claim 1 , wherein the log data in the persistent memory is transferred to a persistent storage device at the remote storage appliance, and the log data is removed from the persistent memory.
9 . A system, comprising:
a processor; and
a memory for holding programmable code; wherein the programmable code includes instructions which, when executed by the processor, cause the processor to execute a set of acts, the set of acts comprising:
executing operations for a first transaction in a first database at a database node of multiple database nodes, wherein
the database node of the multiple database nodes comprises the first database,
the multiple database nodes share use of a persistent memory within a remote storage appliance to hold log records for the first transaction, and
the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective commits of respective transactions executing on the eachrespective database nodes;
generating log data for the operations for the first transaction at the first database;
initiating a commit of the first transaction;
sending the log data together with a request to write the log data to the persistent memory at the remote storage appliance; and
committing the first transaction after successfully sending, by the database node, the request together with the log data for the first transaction using a one-sided protocol to the persistent memory, without the database node waiting for acknowledgment from the remote storage appliance.
10 . The system of claim 9 , wherein the set of acts further comprises:
executing a separate, second transaction that implements a separate uncommitted change to a second database at a second database node of the multiple database nodes, wherein
the second transaction implements an uncommitted change to a first database instance at the database node of the multiple database nodes, and
the first and second transactions are respectively executed on the first and the second database nodes at least by sharing the persistent memory in the remote storage appliance for respective commits of respective first and second transactions.
11 . The system of claim 9 , wherein
the commit is initiated after the first transaction implements an uncommitted change to the first database on the database node,
the remote storage appliance comprises the persistent memory and the persistent storage that includes one or more storage devices, and
a remote direct memory access (RDMA) operation is used to store at least the log data to the persistent memory.
12 . The system of claim 11 , wherein the log records are stored at the persistent memory with the one-sided RDMA protocol that comprises a one-sided RDMA protocol where no response is required from the remote storage appliance.
13 . The system of claim 9 , wherein a zero-copy operation is implemented to store the log data at the persistent memory while bypassing a CPU at the remote storage appliance.
14 . The system of claim 9 , the set of acts further comprising:
after moving the log data for the first transaction to the persistent storage within the remote storage appliance, removing the log data for the first transaction from the persistent memory, wherein the log data is stored at the persistent memory and committed upon a successful remote direct memory access (RDMA) send operation by the database node.
15 . The system of claim 9 , wherein
the first transaction is committed prior to the log data being stored in a persistent storage device,
the request is a one-sided request sent via the one-sided protocol,
the first database sends the request without waiting for the acknowledgement from the remote storage appliance, and
the database node of the multiple database nodes includes one or more database processes to operate over the first database at the database node.
16 . The system of claim 9 , wherein
the log data in the persistent memory is transferred to a persistent storage device at the remote storage appliance, and
the log data is removed from the persistent memory.
17 . A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to perform a set of acts, the set of acts comprising:
executing operations for a first transaction in a first database at a database node of multiple database nodes, wherein
the database node of the multiple database nodes comprises the first database,
the multiple database nodes share use of a persistent memory within a remote storage appliance to hold log records for the first transaction, and
the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective commits of respective transactions executing on respective database nodes;
generating log data for the operations for the first transaction at the first database;
initiating a commit of the first transaction;
sending the log data together with a request to write the log data to the persistent memory at the remote storage appliance; and
committing the first transaction after successfully sending the request, by the database node, together with the log data for the first transaction using a one-sided protocol to the persistent memory, without the database node waiting for acknowledgment from the remote storage appliance.
18 . The computer program product of claim 17 , wherein the set of acts further comprises:
executing a separate, second transaction that implements a separate uncommitted change to a second database at a second database node of the multiple database nodes, wherein
the second transaction implements an uncommitted change to the first database instance at the database node of the multiple database nodes, and
the first and second transactions are respectively executed on the first and the second database nodes at least by sharing the persistent memory in the remote storage appliance for respective commits of respective first and the second transactions.
19 . The computer program product of claim 17 , wherein
the commit is initiated after the first transaction implements an uncommitted change to the first database on the database node,
the remote storage appliance comprises the persistent memory and the persistent storage that includes one or more storage devices, and
a remote direct memory access (RDMA) operation is used to store at least the log data to the persistent memory.
20 . The computer program product of claim 19 , wherein the log data is stored at the persistent memory with the one-sided RDMA protocol that comprises a one-sided RDMA protocol where no response is required from the remote storage appliance.
21 . The computer program product of claim 17 , wherein a zero-copy operation is implemented to store the log data at the persistent memory while bypassing a CPU at the remote storage appliance.
22 . The computer program product of claim 17 , wherein the set of acts further comprises:
after moving the log data for the first transaction to the persistent storage within the remote storage appliance, removing the log data for the first transaction from the persistent memory, wherein the log data is stored at the persistent memory and committed upon a successful remote direct memory access (RDMA) send operation by the database node.
23 . The computer program product of claim 17 , wherein
the first transaction is committed prior to the log data being stored in a persistent storage device,
the request is a one-sided request sent via the one-sided protocol,
the first database sends the request without waiting for the acknowledgement from the remote storage appliance, and
the database node of the multiple database nodes includes one or more database processes to operate over the first database at the database node.
24 . The computer program product of claim 17 , wherein the log data in the persistent memory is transferred to a persistent storage device at the remote storage appliance, and the log data is removed from the persistent memory.
25 . A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to perform a set of acts, the set of acts comprising:
executing operations for a transaction in a database at a database node of multiple database nodes, wherein
a database node of the multiple database nodes comprises the database,
the multiple database nodes share use of a persistent memory in a storage tier of a remote computing device to hold log records for the transaction, wherein
the storage tier of the remote computing device is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for a respective commits of respective transactions executing on respective database nodes;
generating log data for the operations at the database;
initiating a commit of the transaction;
sending the log data together with a request to write the at least the log data to the persistent memory in the storage tier of the remote computing device; and
committing the transaction after a successful send message, by the database node, for successfully sending the request together with the log data for the transaction sent using a one-sided protocol is received, without the database node waiting for acknowledgment from the persistent memory.
26 . The computer program product of claim 25 , wherein
the transaction is committed after receipt of the successful send message and prior to the log data being stored in a persistent storage device,
the request is a one-sided request sent via the one-sided protocol, where the database sends the request without waiting for the acknowledgement from the remote computing device, and
a RDMA (remote direct memory access) operation is used to store the log data to the persistent memory.
27 . The method of claim 1 , wherein the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective fast commits of respective transactions.
28 . The method of claim 2 , wherein the first and second transactions are respectively executed on the first and the second database nodes at least by sharing the persistent memory in the remote storage appliance for respective fast commits of respective first and second transactions.
29 . The system of claim 7 , wherein the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective fast commits of respective transactions.
30 . The system of claim 8 , wherein the first and second transactions are respectively executed on the first and the second database nodes at least by sharing the persistent memory in the remote storage appliance for respective fast commits of respective first and second transactions.
31 . The computer program product of claim 17 , wherein the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective fast commits of respective transactions.
32 . The computer program product of claim 18 , wherein the first and second transactions are respectively executed on the first and the second database nodes at least by sharing the persistent memory in the remote storage appliance for respective fast commits of respective first and second transactions.
33 . The computer program product of claim 25 , wherein the remote storage appliance is connected to and shared by the multiple database nodes over a network so that each database node of the multiple database nodes shares the persistent memory for respective fast commits of respective transactions.