Method to efficiently store object data of an object storage service on a magnetic disk drive and magnetic SMR disk drive
View Patent ↗In a data storage system, the available space of a magnetic storage device is divided into multiple sequential write regions for storing sequentially written data, where the regions are each separated from adjacent sequential write regions by a guard space. Object data, such as key-value pairs, are written sequentially to a particular sequential write region, in blocks of data that correspond to the contents of a nonvolatile buffer being flushed to the magnetic storage device. When a key-value pair stored in the magnetic storage device is subsequently updated, the original key-value pair is not overwritten. Instead, the new version of the key-value pair is included in the next block of data to be written to the magnetic storage device, and a mapping table tracks the location of the newest version of each key-value pair stored in the magnetic storage device.
1. A method of storing data in a data storage system that includes at least one disk drive, the method comprising:
receiving a key-value pair for storage from a client;
storing the received key-value pair in a first non-volatile memory of the data storage system;
prior to storing the received key-value pair in the at least one disk drive, recording in a table stored in a second non-volatile memory of the data storage system, a mapping of a key of the key-value pair received and stored in the first non-volatile memory to a physical location in the at least one disk drive at which the key-value pair is to be stored;
responsive to storing the received key-value pair in the first non-volatile memory and recording the mapping in the table stored in the second non-volatile memory, returning an acknowledgement to the client; and
after storing the received key-value pair in the first non-volatile memory and recording the mapping, storing the received key-value pair in the physical location in the at least one disk drive.
2. The method of claim 1 , further comprising:
receiving one or more additional key-value pairs;
storing the received one or more additional key-value pairs in the first non-volatile memory;
for each of the one or more received additional key-value pairs, recording in the table a mapping of a key of the received additional key-value pair and a respective physical location in the at least one disk drive determined for the received additional key-value pair; and
after storing the one or more additional received key-value pairs in the first non-volatile memory and recording the mapping of the key of the received additional key-value pair and the respective physical location, storing each of the one or more received additional key-value pairs in the respective physical location.
3. The method of claim 2 , further comprising:
prior to storing the received additional key-value pairs in the respective physical locations in the at least one disk drive, determining that a condition for storing the received additional key-value pairs in the respective physical locations in the at least one disk drive is satisfied.
4. The method of claim 3 , wherein the condition is satisfied when a used capacity of the first non-volatile memory is greater than a threshold capacity.
5. The method of claim 4 , further comprising, after storing each of the one or more received additional key-value pairs in the respective physical location in the at least one disk drive, deleting the contents of the first non-volatile memory.
6. The method of claim 3 , further comprising:
upon determining that the condition is satisfied, determining the respective physical locations for the received additional key-value pairs in the at least one disk drive.
7. The method of claim 6 , wherein the physical locations in the at least one disk drive are contiguous in the at least one disk drive.
8. The method of claim 7 , wherein the received key-value pairs are stored in the contiguous physical locations of the at least one disk drive in a single sequential write operation.
9. The method of claim 8 , wherein the at least one disk drive comprises a shingled magnetic recording region of the at least one disk drive.
10. The method of claim 1 , further comprising:
receiving a command from the client to retrieve the value for the received key-value pair;
upon receiving the command from the client, determining whether or not the value for the received key-value pair is stored in the first non-volatile memory;
in response to determining that the value for the received key-value pair is stored in the first non-volatile memory, reading the value for the received key-value pair from the first non-volatile memory; and
in response to determining that the value for the received key-value pair is not stored in the first non-volatile memory, reading the value for the received key-value pair from the physical location in the at least one disk drive.
11. The method of claim 1 , wherein the table associates logical block addresses (LBAs) of the one or more disk drives with a plurality of objects stored on the one or more disk drives.
12. The method of claim 1 , wherein the physical location in the at least one disk drive corresponds to a set of logical block addresses that are associated with an addressable space of the at least one disk drive.
13. The method of claim 1 , further comprising:
after recording the key of the received key-value pair and the physical location in the table, receiving an updated key-value pair for storage, wherein the updated key-value pair includes the same key as the received key-value pair; and
recording in the table the key of the received key-value pair and a new physical location in the at least one disk drive for storing the updated key-value pair.
14. A data storage device comprising:
a support frame that is entirely contained within a region that conforms to a 3.5-inch form-factor disk drive specification;
at least one disk drive mounted on the support frame and entirely contained within the region;
a first non-volatile memory mounted on the support frame and entirely contained within the region;
a second non-volatile memory entirely contained within the region and;
a processor that is entirely contained within the region and is configured to:
receive a key-value pair for storage from a client;
store the received key-value pair in the first non-volatile memory;
prior to storing the received key-value pair in the at least one disk drive, record in a table stored in the second non-volatile memory, a mapping of a key of the key-value pair received and stored in the first non-volatile memory and a physical location in the at least one disk drive at which the key-value pair is to be stored;
responsive to storing the received key-value pair in the first non-volatile memory and recording the mapping in the table stored in the second non-volatile memory, return an acknowledgement to the client; and
in response to a quantity of data stored in the first non-volatile memory exceeding a predetermined value, storing the received key-value pair in the physical location in the at least one disk drive.
15. The data storage device of claim 14 , wherein the at least one disk drive comprises a 2.5-inch form factor disk drive.
16. The data storage device of claim 14 , further comprising a second 2.5-inch form factor disk drive mounted on the support frame and entirely contained within the region.
17. The data storage device of claim 16 , wherein the physical location in the at least one disk drive corresponds to a set of logical block addresses that are associated with an addressable space of the at least one disk drive or the second 2.5-inch form factor disk drive.
18. The data storage device of claim 14 , wherein the physical location in the at least one disk drive corresponds to a set of logical block addresses that are associated with an addressable space of the at least one disk drive.
19. The data storage device of claim 14 , wherein the first non-volatile memory comprises a random-access memory device coupled to a power loss protection circuit.
20. The data storage device of claim 14 , wherein the second non-volatile memory is a flash memory device having a greater storage size and slower access speed than the first non-volatile memory.