Remote direct memory operations (RDMOs) for transactional processing systems
Techniques are described for offloading remote direct memory operations (RDMOs) to “execution candidates”. The execution candidates may be any hardware capable of performing the offloaded operation. Thus, the execution candidates may be network interface controllers, specialized co-processors, FPGAs, etc. The execution candidates may be on a machine that is remote from the processor that is offloading the operation, or may be on the same machine as the processor that is offloading the operation. Details for certain specific RDMOs, which are particularly useful in online transaction processing (OLTP) and hybrid transactional/analytical (HTAP) workloads, are provided.
1. A method comprising:
a process, executing on a first processor, causing a Remote Direct Memory Operation (RDMO) to be executed by an execution candidate that does not include the first processor;
wherein the RDMO comprises a pre-defined sequence of two or more sub-operations;
wherein each sub-operation, of the two or more sub-operations, is one of: a read operation, a write operation, or a compare-and-swap operation; and
wherein causing the RDMO to be executed comprises causing the execution candidate to perform one of:
(a) appending a write value to content of a disk page by:
based, at least in part, on a location value for the content of the disk page, determining whether the disk page includes free space that is at least a size of the write value, and
in response to determining that the disk page includes free space that is at least the size of the write value:
appending the write value to an end of the content of the disk page, and
atomically incrementing the location value by a length of the write value;
(b) traversing one or more pointer-based data structures to identify elements, stored at the one or more pointer-based data structures, that satisfy a particular predicate by, for each pointer-based data structure of the one or more pointer-based data structures:
determining whether a value stored at said each pointer-based data structure satisfies the particular predicate, and
wherein determining whether a particular value stored at a particular pointer-based data structure satisfies the particular predicate comprises comparing, using a comparison operator, the particular value with a parameter value;
wherein the comparison operator is one of: less than, less than or equal to, equal to, greater than or equal to, greater than, or is not equal to;
in response to determining that the value stored at said each pointer-based data structure satisfies the particular predicate, including the value in a results buffer;
(c) reading from a lock-based data structure, associated with a lock, the acquisition of which is required to perform an operation in the lock-based data structure, without performing operations on the lock for the lock-based data structure by:
performing a compare-and-swap operation on a lock location for the lock,
wherein, prior to performing the compare-and-swap operation, the lock location stores an original value,
in response to determining success of said performing the compare-and-swap operation on the lock location:
reading from the lock-based data structure, and
after reading from the lock-based data structure, resetting the lock location to the original value;
(d) copying data to a write location by:
copying data from a buffer to the write location, and
after copying the data from the buffer to the write location, writing a second value at a seal location in memory; or
(e) performing an accumulation operation over one or more values stored at one or more respective addresses by, for each value of the one or more values:
performing the accumulation operation between said each value and a corresponding value in a scatter list to produce an accumulation result value, and
storing the accumulation result value at the address of said each value,
wherein the accumulation operation is one of: FETCHANDADD, SUM, MAX, MIN, COUNT.
2. The method of claim 1 wherein causing an RDMO to be executed involves causing the execution candidate to perform appending a write value to content of a disk page.
3. The method of claim 1 wherein causing an RDMO to be executed involves causing the execution candidate to perform traversing one or more pointer-based data structures to identify elements, stored at the one or more pointer-based data structures, that satisfy a particular predicate.
4. The method of claim 1 wherein causing an RDMO to be executed involves causing the execution candidate to perform reading from a lock-based data structure.
5. The method of claim 1 wherein causing an RDMO to be executed involves causing the execution candidate to perform copying data to a write location.
6. The method of claim 1 wherein causing an RDMO to be executed involves causing the execution candidate to perform an accumulation operation over one or more values stored at one or more respective addresses.
7. The method of claim 1 wherein:
the first processor is on a first computing device;
the execution candidate is on a second computing device that is remote relative to the first computing device; and
causing the RDMO to be executed by the execution candidate comprises sending the RDMO, to the second computing device, via a single network request.
8. The method of claim 1 wherein the first processor and the execution candidate are within a single computing device.
9. The method of claim 1 wherein the execution candidate is a field programmable gate array (FPGA) configured to perform the RDMO.
10. A system comprising:
a first processor executing a process that causes a Remote Direct Memory Operation (RDMO) to be executed by an execution candidate that does not include the first processor;
wherein the RDMO comprises a pre-defined sequence of two or more sub-operations;
wherein each sub-operation, of the two or more sub-operations, is one of: a read operation, a write operation, or a compare-and-swap operation; and
wherein causing the RDMO to be executed comprises causing the execution candidate to perform one of:
(a) appending a write value to content of a disk page by:
based, at least in part, on a location value for the content of the disk page, determining whether the disk page includes free space that is at least a size of the write value, and
in response to determining that the disk page includes free space that is at least the size of the write value:
appending the write value to an end of the content of the disk page, and
atomically incrementing the location value by a length of the write value;
(b) traversing one or more pointer-based data structures to identify elements, stored at the one or more pointer-based data structures, that satisfy a particular predicate by, for each pointer-based data structure of the one or more pointer-based data structures:
determining whether a value stored at said each pointer-based data structure satisfies the particular predicate, and
wherein determining whether a particular value stored at a particular pointer-based data structure satisfies the particular predicate comprises comparing, using a comparison operator, the particular value with a parameter value;
wherein the comparison operator is one of: less than, less than or equal to, equal to, greater than or equal to, greater than, or is not equal to;
in response to determining that the value stored at said each pointer-based data structure satisfies the particular predicate, including the value in a results buffer;
(c) reading from a lock-based data structure, associated with a lock, the acquisition of which is required to perform an operation in the lock-based data structure, without performing operations on the lock for the lock-based data structure by:
performing a compare-and-swap operation on a lock location for the lock,
wherein, prior to performing the compare-and-swap operation, the lock location stores an original value,
in response to determining success of said performing the compare-and-swap operation on the lock location:
reading from the lock-based data structure, and
after reading from the lock-based data structure, resetting the lock location to the original value;
(d) copying data to a write location by:
copying data from a buffer to the write location, and
after copying the data from the buffer to the write location, writing a second value at a seal location in memory; or
(e) performing an accumulation operation over one or more values stored at one or more respective addresses by, for each value of the one or more values:
performing the accumulation operation between said each value and a corresponding value in a scatter list to produce an accumulation result value, and
storing the accumulation result value at the address of said each value,
wherein the accumulation operation is one of: FETCHANDADD, SUM, MAX, MIN, COUNT.
11. The system of claim 10 wherein causing an RDMO to be executed involves causing the execution candidate to perform appending a write value to content of a disk page.
12. The system of claim 10 wherein causing an RDMO to be executed involves causing the execution candidate to perform traversing one or more pointer-based data structures to identify elements, stored at the one or more pointer-based data structures, that satisfy a particular predicate.
13. The system of claim 10 wherein causing an RDMO to be executed involves causing the execution candidate to perform reading from a lock-based data structure.
14. The system of claim 10 wherein causing an RDMO to be executed involves causing the execution candidate to perform copying data to a write location.
15. The system of claim 10 wherein causing an RDMO to be executed involves causing the execution candidate to perform an accumulation operation over one or more values stored at one or more respective addresses.
16. The system of claim 10 wherein:
the first processor is on a first computing device;
the execution candidate is on a second computing device that is remote relative to the first computing device; and
causing the RDMO to be executed by the execution candidate comprises sending the RDMO, to the second computing device, via a single network request.
17. The system of claim 10 wherein the first processor and the execution candidate are within a single computing device.
18. The system of claim 10 wherein the execution candidate is a field programmable gate array (FPGA) configured to perform the RDMO.