Data consistency and durability over distributed persistent memory systems
Examples described herein relates to a network interface apparatus that includes packet processing circuitry and a bus interface. In some examples, the packet processing circuitry to: process a received packet that includes data, a request to perform a write operation to write the data to a cache, and an indicator that the data is to be durable and based at least on the received packet including the request and the indicator, cause the data to be written to the cache and non-volatile memory. In some examples, the packet processing circuitry is to issue a command to an input output (IO) controller to cause the IO controller to write the data to the cache and the non-volatile memory. In some examples, the cache comprises one or more of: a level-0 (L0), level-1 (L1), level-2 (L2), or last level cache (LLC) and the non-volatile memory comprises one or more of: volatile memory that is part of an Asynchronous DRAM Refresh (ADR) domain, persistent memory, battery-backed memory, or memory device whose state is determinate even if power is interrupted to the memory device. In some examples, based on receipt of a second received packet that includes a request to persist data, the packet processing circuitry is to request that data stored in a memory buffer be copied to the non-volatile memory.
1. A network interface apparatus comprising:
packet processing circuitry and a bus interface, the packet processing circuitry to:
process a received packet that includes data, a request to perform a write operation to write the data to a cache, and an indicator that the data is to be durable and
based at least on the received packet including the request and the indicator, cause the data to be written to the cache and non-volatile memory.
2. The apparatus of claim 1 , wherein to cause the data to be written to the cache and the non-volatile memory, the packet processing circuitry is to issue a command to an input output (TO) controller to cause the IO controller to write the data to the cache and the non-volatile memory.
3. The apparatus of claim 1 , wherein:
the cache comprises one or more of: a level-0 (L0), level-1 (L1), level-2 (L2), or last level cache (LLC) and
the non-volatile memory comprises one or more of: volatile memory that is part of an Asynchronous DRAM Refresh (ADR) domain, persistent memory, battery-backed memory, or memory device whose state is determinate even if power is interrupted to the memory device.
4. The apparatus of claim 1 , wherein:
based on receipt of a second received packet that includes a request to persist data, the packet processing circuitry is to request that data stored in a memory buffer be copied to the non-volatile memory.
5. The apparatus of claim 4 , wherein to request that data stored in the memory buffer be copied to the non-volatile memory, the packet processing circuitry is to issue a command to an input output (IO) controller to cause the IO controller to copy the data in the memory buffer into the non-volatile memory.
6. The apparatus of claim 1 , wherein:
based on receipt of a second received packet that includes a request to persist data, the packet processing circuitry is to cause a region of a memory buffer that stores the data, in an event of power loss to the memory buffer, to be copied to a memory device whose state is determinate even if power is interrupted to the memory device.
7. The apparatus of claim 4 , wherein the packet processing circuitry is to:
cause transmission of a response to the request to persist data indicating the data has been persisted regardless of whether the data has been stored in the non-volatile memory.
8. The apparatus of claim 1 , wherein the received packet comprises one or more of: an Ethernet packet, an InfiniBand packet, a remote direct memory access (RDMA) request, or a Data Direct I/O (DDIO) request.
9. The apparatus of claim 1 , comprising one or more of: a host, server, rack, or data center, wherein one or more of the host, server, rack, or data center comprising circuitry to process the data from the cache.
10. A network interface apparatus comprising:
a bus connection and circuitry, wherein the circuitry is to:
form a packet with a command, wherein performance the command to cause a receiver device to copy data directly to a cache and the command includes a flag to cause the receiver device to copy the data also to non-volatile memory and
cause transmission of the packet to the receiver device.
11. The apparatus of claim 10 , wherein the circuitry is to:
form a second packet with a request to persist the data written to a memory buffer to a memory device whose state is determinate even if power is interrupted to the memory device and
cause transmission of the second packet to the receiver device.
12. The apparatus of claim 11 , wherein the circuitry is to:
process a response from the receiver device indicating that the data was stored to the memory device whose state is determinate even if power is interrupted to the memory device.
13. The apparatus of claim 10 , wherein the circuitry is to:
form a second packet with a request to persist the data written to a memory buffer, wherein the request to persist the data is to cause identification of a region of the memory buffer as part of an Asynchronous DRAM Refresh (ADR) domain and
cause transmission of the second packet to the receiver device.
14. The apparatus of claim 10 , wherein the packet comprises one or more of: an Ethernet packet, an InfiniB and packet, a remote direct memory access (RDMA) request, or a Data Direct I/O (DDIO) request.
15. The apparatus of claim 10 , comprising one or more of: a host, server, rack, or data center, wherein one or more of the host, server, rack, or data center are to execute an application that is to cause the form the packet with the command.
16. A method comprising:
a network interface performing:
processing a received packet, wherein the received packet includes: data, a request to perform a write operation to write the data to a cache, and an indicator to make the data durable and
causing the data to be written to the cache and a non-volatile memory.
17. The method of claim 16 , wherein:
the cache comprises one or more of: a level-0 (L0), level-1 (L1), level-2 (L2), or last level cache (LLC) and
the non-volatile memory comprises one or more of: volatile memory that is part of an Asynchronous DRAM Refresh (ADR) domain, persistent memory, battery-backed memory, or memory device whose state is determinate even if power is interrupted to the memory device.
18. The method of claim 16 , comprising:
the network interface performing:
based on receipt of a second received packet that includes a request to persist data, requesting that the data stored in a memory buffer be copied to the non-volatile memory.
19. The method of claim 16 , comprising:
the network interface performing:
based on receipt of a second received packet that includes a request to persist data, causing a region of a memory buffer that stores the data to be part of a region of the memory buffer as part of an Asynchronous DRAM Refresh (ADR) domain.
20. The method of claim 16 , comprising:
the network interface performing:
causing transmission of a response to the indicator to make the data durable indicating the data has been persisted regardless of whether the data has been stored in the non-volatile memory.
21. The method of claim 16 , comprising:
the network interface performing:
causing transmission of a response to the indicator to make the data durable indicating the data has been persisted despite the data not having been stored to the non-volatile memory.
22. The apparatus of claim 1 , wherein indicator that the data is to be durable comprises an indicator to cause copying of the data into non-volatile memory and/or battery-backed volatile memory.
23. The method of claim 16 , wherein the indicator to make the data durable comprises an indicator to cause copying of the data into non-volatile memory and/or battery-backed volatile memory.