Method and system for tiering data

1. A method comprising:

maintaining data in a data storage system having a plurality of tiers, the data storage system including a plurality of volumes with each volume having a mapping table, the mapping table includes: (a) addresses each pointing to a page of data and (b) a reference count indicating a number of times a specific page of data is referenced in the volumes, further wherein the data storage system includes a metadata table having a last write time metadata for each of the addresses in the mapping table;

determining a subset of the data stored in a first, more-durable tier should be stored in a second, less-durable tier, wherein the first, more-durable tier comprises a first all flash storage array, and the second less-durable comprises a second all flash storage array, further wherein the second all flash storage array includes a lower write durability than the first all flash storage array, wherein the determining includes determining the subset of the data from data in a stripe which was previously more than half full, but is currently at least half empty, and determining the subset of the data based on (i) the reference count associated with the subset of the data, and (ii) the last write time metadata associated with the subset of the data; and

tiering the subset of the data from the first, more-durable tier to the second, less-durable tier.

2. The method of claim 1 wherein tiering the subset of the data from the first, more-durable tier to the second, less-durable tier comprises tiering the subset of the data to less expensive flash storage devices comprising the second, less-durable tier.

3. The method of claim 1 wherein determining a subset of the data stored in a first, more-durable tier should be stored in a second, less-durable tier comprises determining the subset of the data has a high fragmentation value.

4. The method of claim 1

wherein the subset of the data is a data block; and

wherein determining the data block stored in a first, more-durable tier should be stored in a second, less-durable tier comprises determining the data block was last written to at least an amount of time in the past satisfying a policy.

5. The method of claim 1

wherein the subset of the data is a range of data blocks; and

wherein determining the range of data blocks stored in a first, more-durable tier should be stored in a second, less-durable tier comprises determining all of the data blocks in the range of data blocks were last written to at least an amount of time in the past satisfying a policy.

6. The method of claim 1

wherein the subset of the data is a base snapshot; and

wherein determining the base snapshot stored in a first, more-durable tier should be stored in a second, less-durable tier comprises determining a lifecycle of the base snapshot as satisfying a policy according to at least one of continuous data protection and a replication engine.

7. The method of claim 1 wherein tiering the subset of the data from the first, more-durable tier to the second, less-durable tier comprises:

receiving a write operation for the subset of the data directed to the first, more-durable tier; and

redirecting the write operation for the subset of the data to the second, less-durable tier.

8. The method of claim 1 further comprising:

monitoring a utilization of the data storage system associated with read I/Os to the data storage system directed to data stored in the second, less-durable tier;

balancing an allocation of data storage system resources across servicing the read I/Os to the data storage system directed to data stored in the second, less-durable tier and write I/Os to the second, less-durable tier associated with the tiering of the subset of the data from the first, more-durable tier to the second, less-durable tier.

9. The method of claim 8 wherein balancing an allocation of data storage system resources across servicing the read I/Os to the data storage system directed to data stored in the second, less-durable tier and write I/Os to the second, less-durable tier associated with the tiering of the subset of the data from the first, more-durable tier to the second, less-durable tier comprises throttling the write I/Os to the second, less-durable tier associated with tiering the subset of the data from the first, more-durable tier to the second, less-durable tier in favor of servicing the read I/Os to the data storage system directed to data stored in the second, less-durable tier of the data storage system.

10. A data storage system comprising:

a first, more-durable tier;

a second, less-durable tier; and

memory storing computer program code that when executed on a processor of the data storage system causes the data storage system to perform the operations of:

maintaining data in the data storage system, the data storage system including a plurality of volumes with each volume having a mapping table, the mapping table includes: (a) addresses each pointing to a page of data and (b) a reference count indicating a number of times a specific page of data is referenced in the volumes, further wherein the data storage system includes a metadata table having a last write time metadata for each of the addresses in the mapping table;

determining a subset of the data having a high value stored in the first, more-durable tier should be stored in a second, less-durable tier, wherein the first, more-durable tier comprises a first all flash storage array, and the second less-durable comprises a second all flash storage array, further wherein the second all flash storage array includes a lower write durability than the first all flash storage array, wherein the determining includes determining the subset of the data from data in a stripe which was previously more than half full, but is currently at least half empty, and determining the subset of the data based on (i) the reference count associated with the subset of the data, and (ii) the last write time metadata associated with the subset of the data; and

tiering the subset of the data having a high value from the first, more-durable tier to the second, less-durable tier.

11. The system of claim 10 wherein computer program code operable to perform the operation of determining a subset of the data stored in a first, more-durable tier should be stored in a second, less-durable tier comprises computer program code operable to perform the operation of determining the subset of the data has a high deduplication ratio.

12. The system of claim 10 wherein computer program code operable to perform the operation of determining a subset of the data stored in a first, more-durable tier should be stored in a second, less-durable tier comprises computer program code operable to perform the operation of determining the subset of the data has a high fragmentation value.

13. The system of claim 10

wherein the subset of the data is a data block; and

wherein computer program code operable to perform the operation of determining the data block stored in a first, more-durable tier should be stored in a second, less-durable tier comprises computer program code operable to perform the operation of determining the data block was last written to at least an amount of time in the past satisfying a policy.

14. The system of claim 10

wherein the subset of the data is a range of data blocks; and

wherein computer program code operable to perform the operation of determining the range of data blocks stored in a first, more-durable tier should be stored in a second, less-durable tier comprises computer program code operable to perform the operation of determining all of the data blocks in the range of data blocks were last written to at least an amount of time in the past satisfying a policy.

15. The system of claim 10

wherein the subset of the data is a base snapshot; and

wherein computer program code operable to perform the operation of determining the base snapshot stored in a first, more-durable tier should be stored in a second, less-durable tier comprises computer program code operable to perform the operation of determining a lifecycle of the base snapshot as satisfying a policy.

16. The system of claim 10 wherein computer program code operable to perform the operation of tiering the subset of the data from the first, more-durable tier to the second, less-durable tier comprises computer program code operable to perform the operations of:

receiving a write operation for the subset of the data directed to the first, more-durable tier; and

redirecting the write operation for the subset of the data to the second, less-durable tier.

17. The system of claim 10 further comprising computer program code operable to perform the operations of:

monitoring a utilization of the data storage system associated with read I/Os to the data storage system directed to data stored in the second, less-durable tier;

balancing an allocation of data storage system resources across servicing the read I/Os to the data storage system directed to data stored in the second, less-durable tier and write I/Os to the second, less-durable tier associated with the tiering of the subset of the data from the first, more-durable tier to the second, less-durable tier.

18. The system of claim 17 wherein computer program code operable to perform the operation of balancing an allocation of data storage system resources across servicing the read I/Os to the data storage system directed to data stored in the second, less-durable tier and write I/Os to the second, less-durable tier associated with the tiering of the subset of the data from the first, more-durable tier to the second, less-durable tier comprises computer program code operable to perform the operation of throttling the write I/Os to the second, less-durable tier associated with tiering the subset of the data from the first, more-durable tier to the second, less-durable tier in favor of servicing the read I/Os to the data storage system directed to data stored in the second, less-durable tier of the data storage system.

19. A computer program product including a non-transitory computer readable storage medium having computer program code encoded thereon that when executed on a processor of a computer causes the computer to execute a tiering controller process, the computer program code comprising:

computer program code for maintaining data in a data storage system having a plurality of tiers, the data storage system including a plurality of volumes with each volume having a mapping table, the mapping table includes: (a) addresses each pointing to a page of data and (b) reference count indicating a number of times a specific page of data is referenced in the volumes, further wherein the data storage system includes a metadata table having a last write time metadata for each of the addresses in the mapping tablet;

computer program code for determining a subset of the data stored in a first, more-durable tier should be stored in a second, less-durable tier, wherein the first, more-durable tier comprises a first all flash storage array, and the second less-durable comprises a second all flash storage array, further wherein the second all flash storage array includes a lower write durability than the first all flash storage array, wherein the determining includes determining the subset of the data from data in a stripe which was previously more than half full, but is currently at least half empty, and determining the subset of the data based on (i) the reference count associated with the subset of the data, and (ii) the last write time metadata associated with the subset of the data; and

computer program code for tiering the subset of the data from the first, more-durable tier to the second, less-durable tier.