Method and system for multi-tier all-flash array
Example embodiments of the present invention relate to a method, a system, and a computer program product for tiering metadata. The method includes selecting a consecutive range of addresses of a logical device having a parent data structure associated therewith maintaining a first set of hash values at a first granularity of the logical device. A second hash value then may be calculated over the consecutive range of addresses of the logical device at a second granularity of the logical device and inserted into a child data structure associated with the parent data structure. Entries in the parent data structure at the first granularity for the consecutive range of addresses then may be freed in favor of the second hash value at the second granularity for the consecutive range of addresses inserted into the child data structure, for storing hash values for other addresses of the logical device.
1. A method comprising:
selecting a consecutive range of addresses of a logical device having a parent data structure associated therewith maintaining a first set of hash values comprising and corresponding to addresses of memory at a first granularity of the logical device, wherein the first set of hash values is generated from contents of the consecutive range of address;
inserting into a child data structure associated with the parent data structure a second hash value calculated over the consecutive range of addresses of the logical device comprising and corresponding to addresses of memory at a second granularity of the logical device, wherein the child data structure requires less storage space than the parent data structure; and
freeing entries in the parent data structure at the first granularity for the consecutive range of addresses, in favor of the second hash value at the second granularity for the consecutive range of addresses inserted into the child data structure, for storing hash values for other addresses of the logical device.
2. The method of claim 1 further comprising maintaining metadata in memory for the logical device according to the parent data structure maintaining the first set of hash values at the first granularity of the logical device and according to the child data structure into which was inserted the second hash value at the second granularity of the logical device.
3. The method of claim 2 further comprising performing inline deduplication over the logical device at both the first granularity and the second granularity.
4. The method of claim 3 wherein performing inline deduplication over the logical device at both the first granularity and the second granularity comprises:
performing a first inline deduplication of a first portion of the logical device according to the first set of hash values for the logical device in the parent data structure at the first granularity; and
performing a second inline deduplication of a second portion of the logical device according to the second hash value for the logical device in the child data structure at the second granularity.
5. The method of claim 1 further comprising serving Input/Output (I/O) to the logical device according to the parent data structure at the first granularity and according to the child data structure at the second granularity.
6. The method of claim 5 wherein serving Input/Output (I/O) to the logical device according to the parent data structure at the first granularity and according to the child data structure at the second granularity comprises, for a write I/O to the logical device:
writing data of the write I/O to its target address; and
writing a hash value for the write I/O to the parent data structure associated with the logical device.
7. The method of claim 5 wherein serving Input/Output (I/O) to the logical device according to the parent data structure at the first granularity and according to the child data structure at the second granularity comprises, for a read I/O to an address of the logical device:
attempting a first read of a hash value associated with the address from the parent data structure at the first granularity;
if the first read succeeds, returning the read hash value from the parent data structure at the first granularity;
if the first read fails, attempting a second read of the hash value associated with the address from the child data structure at the second granularity;
if the second read succeeds, returning the read hash value from the child data structure at the second granularity; and
if the second read fails, return a status.
8. The method of claim 7 further comprising using the read hash value as an index into a third data structure to identify a location of data satisfying the I/O on physical disks.
9. The method of claim 1 wherein selecting a consecutive range of addresses of a logical device having a parent data structure associated therewith maintaining a first set of hash values at a first granularity of the logical device comprises selecting the consecutive range of addresses according to a heat map identifying a frequency of access for each address.
10. The method of claim 9 further comprising tiering storage of the logical device by migrating data associated with the consecutive range of addresses from a first storage tier to a second storage tier.
11. A system comprising:
a processor;
a storage array;
a logical device stored on one or more storage devices on the storage array; and
computer program code that when executed on the processor performs the operations of:
selecting a consecutive range of addresses of the logical device having a parent data structure associated therewith maintaining a first set of hash values comprising and corresponding to addresses of memory at a first granularity of the logical device, wherein the first set of hash values is generated from contents of the consecutive range of address;
inserting into a child data structure associated with the parent data structure a second hash value calculated over the consecutive range of addresses comprising and corresponding to addresses of memory of the logical device at a second granularity of the logical device; and
freeing entries in the parent data structure at the first granularity for the consecutive range of addresses, in favor of the second hash value at the second granularity for the consecutive range of addresses inserted into the child data structure, for storing hash values for other addresses of the logical device, wherein the child data structure requires less storage space than the parent data structure.
12. The system of claim 11 further comprising maintaining metadata in memory for the logical device according to the parent data structure maintaining the first set of hash values at the first granularity of the logical device and according to the child data structure into which was inserted the second hash value at the second granularity of the logical device.
13. The system of claim 12 further comprising performing inline deduplication over the logical device at both the first granularity and the second granularity.
14. The system of claim 13 wherein performing inline deduplication over the logical device at both the first granularity and the second granularity comprises:
performing a first inline deduplication of a first portion of the logical device according to the first set of hash values for the logical device in the parent data structure at the first granularity; and
performing a second inline deduplication of a second portion of the logical device according to the second hash value for the logical device in the child data structure at the second granularity.
15. The system of claim 11 further comprising serving Input/Output (I/O) to the logical device according to the parent data structure at the first granularity and according to the child data structure at the second granularity.
16. The system of claim 15 wherein serving Input/Output (I/O) to the logical device according to the parent data structure at the first granularity and according to the child data structure at the second granularity comprises, for a write I/O to the logical device:
writing data of the write I/O to its target address; and
writing a hash value for the write I/O to the parent data structure associated with the logical device.
17. The system of claim 15 wherein serving Input/Output (I/O) to the logical device according to the parent data structure at the first granularity and according to the child data structure at the second granularity comprises, for a read I/O to an address of the logical device:
attempting a first read of a hash value associated with the address from the child data structure at the second granularity;
if the first read succeeds, returning the read hash value from the child data structure at the second granularity;
if the first read fails, attempting a second read of the hash value associated with the address from the parent data structure at the first granularity;
if the second read succeeds, returning the read hash value from the parent data structure at the first granularity; and
if the second read fails, failing the read I/O.
18. The system of claim 17 further comprising using the read hash value as an index into a third data structure to identify a location of data satisfying the I/O on physical disks.
19. The system of claim 11 wherein selecting a consecutive range of addresses of a logical device having a parent data structure associated therewith maintaining a first set of hash values at a first granularity of the logical device comprises selecting the consecutive range of addresses according to a heat map identifying a frequency of access for each address.
20. The system of claim 19 further comprising tiering storage of the logical device by migrating data associated with the consecutive range of addresses from a first storage tier to a second storage tier.
21. A computer program product having computer executable code embodied in non-transitory storage media comprising:
computer program code for selecting a consecutive range of addresses of a logical device having a parent data structure associated therewith maintaining a first set of hash values comprising and corresponding to addresses of memory at a first granularity of the logical device, wherein the first set of hash values is generated from contents of the consecutive range of address;
computer program code for inserting into a child data structure associated with the parent data structure a second hash value calculated over the consecutive range of addresses comprising and corresponding to addresses of memory of the logical device at a second granularity of the logical device; and
computer program code for freeing entries in the parent data structure at the first granularity for the consecutive range of addresses, in favor of the second hash value at the second granularity for the consecutive range of addresses inserted into the child data structure, for storing hash values for other addresses of the logical device, wherein the child data structure requires less storage space than the parent data structure.