System and method for data organization in shingled magnetic recording drive
One embodiment described herein provides a system for managing metadata for a shingled magnetic recording (SMR) drive. During operation, the system stores a metadata file in a memory of a host of the SMR drive; subsequent to the SMR drive performing a write operation, the system generates a metadata update, and sends the generated metadata update to a buffer associated with the SMR drive and the memory of the host. In response to a compaction condition being met, the system launches, in the memory of the host, a metadata compaction operation to generate a compacted metadata file based on the previously stored metadata file and the metadata update, and sends the compacted metadata file to the SMR drive.
1. A computer-implemented method for managing metadata for a shingled magnetic recording (SMR) drive, the method comprising:
storing a metadata file in a memory of a host of the SMR drive;
subsequent to the SMR drive performing a write operation, generating a metadata update;
sending the generated metadata update to a buffer located on the SMR drive and the memory of the host;
in response to a compaction condition being met, launching, by the memory of the host, a metadata compaction operation to generate a compacted metadata file based on the previously stored metadata file and the metadata update; and
sending the compacted metadata file to the SMR drive.
2. The computer-implemented method of claim 1 , wherein the compaction condition comprises one or more of:
an occurrence of a timeout event;
a size of the metadata update exceeding a predetermined threshold; and
receiving a command from the host of the SMR drive.
3. The computer-implemented method of claim 1 , wherein the stored metadata file comprises a log file, and wherein generating the metadata update comprises generating a log entry.
4. The computer-implemented method of claim 3 , wherein the generated log entry is appended to the log file.
5. The computer-implemented method of claim 1 , further comprising:
creating, by a file system associated with the SMR drive, a logical extent corresponding to a data bucket;
mapping the logical extent to a physical extent on the SMR drive; and
saving the mapping between the logical extent and the physical extent in the metadata update.
6. The computer-implemented method of claim 5 , wherein the physical extent corresponds to a continuous physical space on the SMR drive.
7. The computer-implemented method of claim 6 , wherein the continuous physical space is aligned to an SMR zone.
8. The computer-implemented method of claim 1 , wherein the metadata compaction operation comprises multiple incremental compacting operations.
9. The computer-implemented method of claim 1 , further comprising:
in response to detecting a power loss, writing the metadata update stored in the buffer to a flash drive associated with the SMR drive.
10. The computer-implemented method of claim 9 , further comprising:
in response to detecting a power recovery, writing the metadata update stored in the flash drive back to the buffer; and
loading the compacted metadata file stored on the SMR drive to the memory of the host.
11. A shingled magnetic recording (SMR)-based data storage system, comprising:
an SMR drive;
a host server comprising a memory configured to store a metadata file associated with the SMR drive; and
a buffer located on the SMR drive, wherein the buffer is configured to store updates to the metadata file;
wherein the memory of the host server is configured to, in response to a compaction condition being met, launch a metadata compaction operation to generate a compacted metadata file based on the previously stored metadata file and the updates to the metadata file; and
wherein the SMR drive is configured to receive and store the compacted metadata file.
12. The data storage system of claim 11 , wherein the compaction condition comprises one or more of:
an occurrence of a timeout event;
a size of the metadata update exceeding a predetermined threshold; and
receiving a command from the host of the SMR drive.
13. The data storage system of claim 11 , wherein the stored metadata file comprises a log file, and wherein the metadata updates comprise log entries.
14. The data storage system of claim 13 , wherein the generated log entry is appended to the log file.
15. The data storage system of claim 11 , further comprising:
a file system associated with the SMR drive configured to:
create a logical extent corresponding to a data bucket;
map the logical extent to a physical extent on the SMR drive; and
save the mapping between the logical extent and the physical extent in the updates to the metadata file.
16. The data storage system of claim 15 , wherein the physical extent corresponds to a continuous physical space on the SMR drive.
17. The data storage system of claim 16 , wherein the continuous physical space is aligned to an SMR zone.
18. The data storage system of claim 11 , wherein the metadata compaction operation comprises multiple incremental compacting operations.
19. The data storage system of claim 11 , further comprising:
a flash drive associated with the SMR drive; and
a metadata backup module configured to, in response to detecting a power loss, back up the updates to the metadata file stored in the buffer to the flash drive.
20. The data storage system of claim 19 , wherein the metadata backup module is further configured to:
in response to detecting a power recovery, write the updates to the metadata file stored in the flash drive back to the buffer; and
load the compacted metadata file stored on the SMR drive to the memory of the host.