Methods for early write termination into non-volatile memory with metadata write operations
View Patent ↗In one embodiment of the invention, a memory apparatus for improved write performance is disclosed. The memory apparatus includes a base printed circuit board (PCB) having an edge connector for plugging into a host server system; a card level power source to provide card level power during a power failure; a memory controller coupled to the card level power source and having one or more memory channels; and one or more non-volatile memory devices (NVMDs) coupled to the card level power source and organized to respectively couple to the memory channels controlled by the memory controller. Each memory controller provides queuing and scheduling of memory operations on a channel for each NVMD in the memory channels. Responsive to power failure, the memory controller receives card level power and changes the scheduling of memory operations to the NVMDs in each memory channel.
1. A method for writing data into non-volatile memory in an expansion memory card with a software driver, the method comprising:
issuing a first data write operation from operating system software to a software driver for first write data;
transferring the first write data to a volatile memory buffer within a memory controller controlling access to one or more non-volatile memory devices (NVMDs) organized in one or more memory channels;
initiating the first data write operation to one of one or more channel controllers in the memory controller, wherein each of the one or more channel controllers to control one or more NVMDs organized on the same memory channel;
issuing a second metadata write operation from the operating system software to the software driver for second write data;
in response to the second metadata write operation, storing a first delayed write request in a channel controller of the memory controller as a first stored delayed write request, wherein the first stored delayed write request identifies a starting address of a delayed write buffer in a volatile memory in the memory controller and wherein the first stored delayed write request further identifies a first starting NVMD address to which the second write data is to be stored in a memory channel comprising one or more NVMDs;
transferring the second write data associated with the second metadata write operation to the delayed write buffer; and
before completion of the first data write operation and the second metadata write operation to one or more NVMDs, notifying the operating system software that the first data write operation and the second metadata write operation are guaranteed to be completed with a probability greater than fifty percent, even if a power failure occurs thereafter.
2. The method of claim 1 , further comprising:
issuing a third data write operation from the operating system software to the software driver for third write data;
transferring the third write data to the volatile memory buffer within the memory controller;
initiating the third data write operation to one of the one or more channel controllers in the memory controller;
issuing a fourth metadata write operation from the operating system software to the software driver for fourth write data;
transferring the fourth write data associated with the fourth metadata write operation to an unused memory location in the delayed write buffer;
in response to the fourth metadata write operation, storing a second delayed write request in the channel controller of the memory controller as a second stored delayed write request, wherein the second stored delayed write request identifies a starting address of the delayed write buffer in the volatile memory in the memory controller and wherein the second stored delayed write request further identifies a first starting NVMD address to which the fourth write data is to be stored in the memory channel comprising the one or more NVMDs; and
before completion of the third data write operation and the fourth metadata write operation to one or more NVMDs, notifying the operating system software that the third data write operation and the fourth metadata write operation are guaranteed to be completed with a probability greater than fifty percent, even if a power failure occurs thereafter.
3. The method of claim 2 , wherein
the expansion memory card includes one or more flash memory devices,
the software driver provides a block device view of the flash memory devices; and
the fourth metadata write operation further stores a translation between a logical block number in the block device view and a physical block address identifying a unique memory block in the flash memory devices.
4. The method of claim 2 , wherein
if a power failure occurs, the first and second stored delayed write requests are guaranteed to be completed by
switching to a card level power source on the expansion memory card, and
completing any pending delayed write requests for writing data from the delayed write buffer into an NVMD beginning with the first starting NVMD address.
5. The method of claim 1 , wherein
the expansion memory card includes one or more flash memory devices,
the software driver provides a block device view of the flash memory devices; and
the second metadata write operation further stores a translation between a logical block number in the block device view and a physical block address identifying a unique memory block in the flash memory devices.
6. The method of claim 1 , wherein
if a power failure occurs, the first stored delayed write request is guaranteed to be completed by
switching to a card level power source on the expansion memory card, and
completing any pending delayed write requests for writing data from the delayed write buffer into an NVMD beginning with the first starting NVMD address.
7. The method of claim 6 , further comprising:
prior to completing any pending delayed write requests, completing execution of any of the currently executing operations with NVMDs in the channel.
8. The method of claim 7 , wherein
if a power failure occurs, the second metadata write operation is further guaranteed to be completed by
initiating the first stored delayed write request on the channel to write a second metadata write data from the delayed write buffer to a first non-volatile memory (NVM) location at a first NVMD address in a first memory controller channel; and
waiting for completion of the first stored delayed write request.
9. The method of claim 8 , wherein the second metadata write operation is further guaranteed to be completed by
examining results of the first stored delayed write request;
if the first stored delayed write request was unsuccessful, then attempting a first rewrite operation to rewrite the second metadata write data from the delayed write buffer into a second NVM location at a first NVMD backup address provided by a first backup register in the first memory channel controller.
10. The method of claim 9 , wherein the second metadata write operation is further guaranteed to be completed by
examining results of the first rewrite operation;
if the first rewrite operation was unsuccessful, then attempting a second rewrite operation to rewrite the second metadata write data from the delayed write buffer into a third NVM location at a second NVMD backup address provided by a second backup register in the first memory channel controller.
11. The method of claim 10 , wherein the second metadata write operation is further guaranteed to be completed by
examining results of the second rewrite operation;
if the second rewrite operation was unsuccessful, then attempting a third rewrite operation to rewrite the second metadata write data from the delayed write buffer into a fourth NVM location at a third NVMD backup address provided by a third backup register in the first memory channel controller.
12. The method of claim 1 , wherein
the notification of the guaranteed completion is issued immediately after the initiating of a first write operation associated with the first data write operation and the storing of the first stored delayed write request associated with the second metadata write operation into the NVMD.
13. The method of claim 1 , wherein
if a power failure occurs, the first data write operation is guaranteed to be completed by
switching to a card level power source on the expansion memory card, and
completing any current write operations of write data from the volatile memory buffer to NVMDs.
14. The method of claim 13 , wherein
if a power failure occurs, the first data write operation is further guaranteed to be completed by
waiting for completion of the first data write operation to write a first write data from the volatile memory buffer to a first NVM location at a first NVMD address in a first memory controller channel;
examining results of the first data write operation; and
if the first data write operation was unsuccessful, then attempting a first rewrite operation to rewrite the first write data from the volatile memory buffer into a second NVM location at a first NVMD backup address provided by a first backup register in the first memory channel controller.
15. The method of claim 14 , further comprising:
examining results of the first rewrite operation; and
if the first rewrite operation was unsuccessful, then attempting a second rewrite operation to rewrite the first write data from the volatile memory buffer into a third NVM location at a second NVMD backup address provided by a second backup register in the first memory channel controller.
16. The method of claim 15 , further comprising:
examining results of the second rewrite operation; and
if the second rewrite operation was unsuccessful, then attempting a third rewrite operation to rewrite the first write data from the volatile memory buffer into a fourth NVM location at a third NVMD backup address provided by a third backup register in the first memory channel controller.
17. The method of claim 1 , wherein
the expansion memory card is included in a computer system; and
the operating system software is notified that the first data write operation and the second metadata write operation are to be completed with a probability greater than ninety-five percent, even if a power failure occurs thereafter.
18. The method of claim 1 , wherein
the expansion memory card is included in a server system; and
the operating system software is notified that the first data write operation and the second metadata write operation are to be completed with a probability greater than ninety-nine percent, even if a power failure occurs thereafter.
19. A method for writing data into non-volatile memory in an expansion memory card with a software driver, the method comprising:
issuing a first data write operation from application software to a software driver for first write data;
transferring the first write data to a volatile memory buffer within a memory controller controlling access to one or more non-volatile memory devices (NVMDs) organized in one or more memory channels;
initiating the first data write operation to one of one or more channel controllers in the memory controller, wherein each of the one or more channel controllers to control one or more NVMDs organized on the same memory channel;
issuing a second metadata write operation from the application software to the software driver for second write data;
in response to the second metadata write operation, storing a first delayed write request in a channel controller of the memory controller as a first stored delayed write request, wherein the first stored delayed write request identifies a starting address of a delayed write buffer in a volatile memory in the memory controller and wherein the first stored delayed write request further identifies a first starting NVMD address to which the second write data is to be stored in a memory channel comprising one or more NVMDs;
transferring the second write data associated with the second metadata write operation to the delayed write buffer; and
before completion of the first data write operation and the second metadata write operation to one or more NVMDs, notifying the application software that the first data write operation and the second metadata write operation are guaranteed to be completed with a probability greater than fifty percent, even if a power failure occurs thereafter.
20. The method of claim 19 , further comprising:
issuing a third data write operation from the application software to the software driver for third write data;
transferring the third write data to the volatile memory buffer within the memory controller;
initiating the third data write operation to one of the one or more channel controllers in the memory controller;
issuing a fourth metadata write operation from the application software to the software driver for fourth write data;
transferring the fourth write data associated with the fourth metadata write operation to an unused memory location in the delayed write buffer;
in response to the fourth metadata write operation, storing a second delayed write request in the channel controller of the memory controller as a second stored delayed write request, wherein the second stored delayed write request identifies a starting address of the delayed write buffer in the volatile memory in the memory controller and wherein the second stored delayed write request further identifies a first starting NVMD address to which the fourth write data is to be stored in the memory channel comprising the one or more NVMDs; and
before completion of the third data write operation and the fourth metadata write operation to one or more NVMDs, notifying the application software that the third data write operation and the fourth metadata write operation are guaranteed to be completed with a probability greater than fifty percent, even if a power failure occurs thereafter.
21. The method of claim 20 , wherein
the expansion memory card includes one or more flash memory devices,
the software driver provides a block device view of the flash memory devices; and
the fourth metadata write operation further stores a translation between a logical block number in the block device view and a physical block address identifying a unique memory block in the flash memory devices.
22. The method of claim 20 , wherein
if a power failure occurs, the first and second stored delayed write requests are guaranteed to be completed by
switching to a card level power source on the expansion memory card, and
completing any pending delayed write requests for writing data from the delayed write buffer into an NVMD beginning with the first starting NVMD address.
23. The method of claim 19 , wherein
the expansion memory card includes one or more flash memory devices,
the software driver provides a block device view of the flash memory devices; and
the second metadata write operation further stores a translation between a logical block number in the block device view and a physical block address identifying a unique memory block in the flash memory devices.
24. The method of claim 19 , wherein
if a power failure occurs, the first stored delayed write request is guaranteed to be completed by
switching to a card level power source on the expansion memory card, and
completing any pending delayed write requests for writing data from the delayed write buffer into an NVMD beginning with the first starting NVMD address.
25. The method of claim 24 , further comprising:
prior to completing any pending delayed write requests, completing execution of any of the currently executing operations with NVMDs in the channel.
26. The method of claim 25 , wherein
if a power failure occurs, the second metadata write operation is further guaranteed to be completed by
initiating the first stored delayed write request on the channel to write a second metadata write data from the delayed write buffer to a first non-volatile memory (NVM) location at a first NVMD address in a first memory controller channel; and
waiting for completion of the first stored delayed write request.
27. The method of claim 26 , wherein the second metadata write operation is further guaranteed to be completed by
examining results of the first stored delayed write request;
if the first stored delayed write request was unsuccessful, then attempting a first rewrite operation to rewrite the second metadata write data from the delayed write buffer into a second NVM location at a first NVMD backup address provided by a first backup register in the first memory channel controller.
28. The method of claim 27 , wherein the second metadata write operation is further guaranteed to be completed by
examining results of the first rewrite operation;
if the first rewrite operation was unsuccessful, then attempting a second rewrite operation to rewrite the second metadata write data from the delayed write buffer into a third NVM location at a second NVMD backup address provided by a second backup register in the first memory channel controller.
29. The method of claim 28 , wherein the second metadata write operation is further guaranteed to be completed by
examining results of the second rewrite operation;
if the second rewrite operation was unsuccessful, then attempting a third rewrite operation to rewrite the second metadata write data from the delayed write buffer into a fourth NVM location at a third NVMD backup address provided by a third backup register in the first memory channel controller.
30. The method of claim 19 , wherein
the notification of the guaranteed completion is issued immediately after the initiating of a first write operation associated with the first data write operation and the storing of the first stored delayed write request associated with the second metadata write operation into the NVMD.
31. The method of claim 19 , wherein
if a power failure occurs, the first data write operation is guaranteed to be completed by
switching to a card level power source on the expansion memory card, and
completing any current write operations of write data from the volatile memory buffer to NVMDs.
32. The method of claim 31 , wherein
if a power failure occurs, the first data write operation is further guaranteed to be completed by
waiting for completion of the first data write operation to write a first write data from the volatile memory buffer to a first NVM location at a first NVMD address in a first memory controller channel;
examining results of the first write operation; and
if the first write operation was unsuccessful, then attempting a first rewrite operation to rewrite the first write data from the volatile memory buffer into a second NVM location at a first NVMD backup address provided by a first backup register in the first memory channel controller.
33. The method of claim 32 , further comprising:
examining results of the first rewrite operation; and
if the first rewrite operation was unsuccessful, then attempting a second rewrite operation to rewrite the first write data from the volatile memory buffer into a third NVM location at a second NVMD backup address provided by a second backup register in the first memory channel controller.
34. The method of claim 33 , further comprising:
examining results of the second rewrite operation; and
if the second rewrite operation was unsuccessful, then attempting a third rewrite operation to rewrite the first write data from the volatile memory buffer into a fourth NVM location at a third NVMD backup address provided by a third backup register in the first memory channel controller.
35. The method of claim 19 , wherein
the expansion memory card is included in a computer system; and
the application software is notified that the first data write operation and the second metadata write operation are to be completed with a probability greater than ninety-five percent, even if a power failure occurs thereafter.
36. The method of claim 19 , wherein
the expansion memory card is included in a server system; and
the application software is notified that the first data write operation and the second metadata write operation are to be completed with a probability greater than ninety-nine percent, even if a power failure occurs thereafter.