Persistent memory management
Apparatuses, systems, methods, and computer program products are disclosed for persistent memory management. Persistent memory management may include providing a persistent data structure stored at least partially in volatile memory configured to ensure persistence of the data structure in a non-volatile memory medium. Persistent memory management may include replicating a persistent data structure in volatile memory buffers of at least two non-volatile storage devices. Persistent memory management may include preserving a snapshot copy of data in association with completion of a barrier operation for the data. Persistent memory management may include determining which interface of a plurality of supported interfaces is to be used to flush data from a processor complex.
1. A method comprising:
allocating a plurality of auto-commit memory buffers for storing a persistent data structure, in response to receiving a request from a client, the request comprising a logical identifier in a namespace for persistent data structures, wherein:
an auto-commit memory comprises a controller, a non-volatile memory medium, and the allocated auto-commit memory buffers,
a host is a computing device comprising a bus,
the auto-commit memory is a hardware device with a single connection to the bus, and
the auto-commit memory buffers comprise uniformly sized regions of a volatile memory;
arming the allocated auto-commit memory buffers by storing metadata in the allocated auto-commit memory buffers such that the metadata specifies addresses of the non-volatile memory medium for storing data of the persistent data structure, and such that individual auto-commit memory buffers store their own per-buffer portions of the metadata specifying where to store their own per-buffer portions of the data;
writing data for a first end of the persistent data structure from the host to the allocated auto-commit memory buffers, the persistent data structure comprising a queue data structure wherein the first end of the persistent data structure differs from a second end of the persistent data structure;
storing at least a portion of the data of the persistent data structure in the non-volatile memory medium;
loading data of the second end of the persistent data structure from the non-volatile memory medium into the volatile memory for access by the client; and
in response to a trigger, using the controller to store data of the first end of the persistent data structure from the allocated auto-commit memory buffers to the addresses specified by the metadata, wherein storing the data of the first end of the persistent data structure is internal to the hardware device without communicating via the single connection.
2. The method of claim 1 , further comprising enforcing one or more rules for the persistent data structure.
3. The method of claim 2 , wherein:
the one or more rules define the queue data structure as sequential and append-only; and
enforcing the one or more rules comprises requiring that data be written only to the first end of the queue data structure and that data be read only from the second end of the queue data structure.
4. The method of claim 1 , wherein the portion of the data stored in the non-volatile memory medium is between the first end and the second end of the persistent data structure.
5. The method of claim 1 , further comprising tracking which data of the persistent data structure resides in the volatile memory and which data of the persistent data structure resides in the non-volatile memory medium.
6. The method of claim 1 , further comprising limiting a rate at which the data for the first end of the persistent data structure is written to the volatile memory such that the rate is at, or below, a rate at which data of the persistent data structure is copied to the non-volatile memory medium.
7. The method of claim 1 , wherein an isolation zone of the auto-commit memory comprises the controller, the non-volatile storage medium, the auto-commit memory buffers, and a secondary power source configured to power the controller and the auto-commit memory buffers despite failure of the host.
8. An apparatus comprising:
means for allocating a plurality of auto-commit memory buffers for storing a persistent data structure, in response to receiving a request from a client, the request comprising a logical identifier in a namespace for persistent data structures, wherein:
an auto-commit memory comprises a controller, a non-volatile memory medium, and the allocated auto-commit memory buffers,
a host is a computing device comprising a bus,
the auto-commit memory is a hardware device with a single connection to the bus, and
the auto-commit memory buffers comprise uniformly sized regions of a volatile memory;
means for arming the allocated auto-commit memory buffers by storing metadata in the allocated auto-commit memory buffers such that the metadata specifies addresses of the non-volatile memory medium for storing data of the persistent data structure, and such that individual auto-commit memory buffers store their own per-buffer portions of the metadata specifying where to store their own per-buffer portions of the data;
means for writing data for a first end of the persistent data structure from the host to the allocated auto-commit memory buffers, the persistent data structure comprising a queue data structure wherein the first end of the persistent data structure differs from a second end of the persistent data structure;
means for storing at least a portion of the data of the persistent data structure in the non-volatile memory medium;
means for loading data of the second end of the persistent data structure from the non-volatile memory medium into the volatile memory for access by the client; and
means for using the controller, in response to a trigger, to store data of the first end of the persistent data structure from the allocated auto-commit memory buffers to the addresses specified by the metadata, wherein storing the data of the first end of the persistent data structure is internal to the hardware device without communicating via the single connection.
9. The apparatus of claim 8 , further comprising means for enforcing one or more rules for the persistent data structure.
10. The apparatus of claim 9 , wherein:
the one or more rules define the queue data structure as sequential and append-only; and
enforcing the one or more rules comprises requiring that data be written only to the first end of the queue data structure and that data be read only from the second end of the queue data structure.
11. The apparatus of claim 8 , wherein the portion of the data stored in the non-volatile memory medium is between the first end and the second end of the persistent data structure.
12. The apparatus of claim 8 , further comprising means for maintaining an address mapping structure, the address mapping structure tracking which data of the persistent data structure resides in the volatile memory and which data of the persistent data structure resides in the non-volatile memory medium.
13. The apparatus of claim 8 , further comprising means for limiting a rate at which the data for the first end of the persistent data structure is written to the volatile memory such that the rate is at, or below, a rate at which data of the persistent data structure is copied to the non-volatile memory medium.
14. A computer program product comprising a non-transitory computer readable medium storing computer usable program code executable to perform operations, the operations comprising:
allocating a plurality of auto-commit memory buffers for storing a persistent data structure, in response to receiving a request from a client, the request comprising a logical identifier in a namespace for persistent data structures, wherein:
an auto-commit memory comprises a controller, a non-volatile memory medium, and the allocated auto-commit memory buffers,
a host is a computing device comprising a bus,
the auto-commit memory is a hardware device with a single connection to the bus, and
the auto-commit memory buffers comprise uniformly sized regions of a volatile memory;
arming the allocated auto-commit memory buffers by storing metadata in the allocated auto-commit memory buffers such that the metadata specifies addresses of the non-volatile memory medium for storing data of the persistent data structure, and such that individual auto-commit memory buffers store their own per-buffer portions of the metadata specifying where to store their own per-buffer portions of the data;
writing data for a first end of the persistent data structure from the host to the allocated auto-commit memory buffers, the persistent data structure comprising a queue data structure wherein the first end of the persistent data structure differs from a second end of the persistent data structure;
storing at least a portion of the data of the persistent data structure in the non-volatile memory medium;
loading data of the second end of the persistent data structure from the non-volatile memory medium into the volatile memory for access by the client; and
in response to a trigger, using the controller to store data of the first end of the persistent data structure from the allocated auto-commit memory buffers to the addresses specified by the metadata, wherein storing the data of the first end of the persistent data structure is internal to the hardware device without communicating via the single connection.
15. The computer program product of claim 14 , the operations further comprising enforcing one or more rules for the persistent data structure.
16. The computer program product of claim 15 , wherein:
the one or more rules define the queue data structure as sequential and append-only; and
enforcing the one or more rules comprises requiring that data be written only to the first end of the queue data structure and that data be read only from the second end of the queue data structure.
17. The computer program product of claim 14 , wherein the portion of the data stored in the non-volatile memory medium is between the first end and the second end of the persistent data structure.
18. The computer program product of claim 14 , the operations further comprising tracking which data of the persistent data structure resides in the volatile memory and which data of the persistent data structure resides in the non-volatile memory medium.
19. The computer program product of claim 14 , the operations further comprising limiting a rate at which the data for the first end of the persistent data structure is written to the volatile memory such that the rate is at, or below, a rate at which data of the persistent data structure is copied to the non-volatile memory medium.
20. The computer program product of claim 14 , the operations further comprising receiving a barrier request for the volatile memory.
21. A system comprising:
a host device comprising a peripheral bus, wherein the host device is a computing device;
a data storage device having a single connection to the peripheral bus, the data storage device comprising a volatile memory, a non-volatile storage medium, and a secondary power supply; and
a controller comprising:
an allocation module configured to allocate a plurality of auto-commit memory buffers for storing a persistent data structure, in response to receiving a request from a client, the request comprising a logical identifier in a namespace for persistent data structures, the auto-commit memory buffers comprising uniformly sized regions of the volatile memory,
a storage management module configured to arm the allocated auto-commit memory buffers by storing metadata in the allocated auto-commit memory buffers such that the metadata specifies addresses of the non-volatile storage medium for storing data of the persistent data structure, and such that individual auto-commit memory buffers store their own per-buffer portions of the metadata specifying where to store their own per-buffer portions of the data,
a write module configured to write data for a first end of the persistent data structure from the host device to the allocated auto-commit memory buffers, the persistent data structure comprising a queue data structure wherein the first end of the persistent data structure differs from a second end of the persistent data structure,
a destage module configured to store at least a portion of the data of the persistent data structure in the non-volatile storage medium,
a read module configured to load data of the second end of the persistent data structure from the non-volatile storage medium into the volatile memory for access by the client, and
a commit module configured to store data of the first end of the persistent data structure from the allocated auto-commit memory buffers to the addresses specified by the metadata, in response to a trigger, wherein storing the data of the first end of the persistent data structure is internal to the data storage device without communicating via the single connection to the peripheral bus,
wherein the allocation module, the storage management module, the write module, the destage module, the read module, and the commit module comprise one or more of logic hardware and executable code, the executable code stored on a non-transitory computer readable medium.
22. The system of claim 21 , wherein the controller comprises a hardware controller for the data storage device.