Multi-tiered data storage in a deduplication system
Systems, methods, and devices are disclosed herein for data deduplication and storage. Methods include identifying first criteria associated with a data block included in a plurality of data blocks, the data block being identified based on a blockmap and an index, the blockmap characterizing a file stored in a deduplication repository, the index identifying data store identifiers, reference count data, and second criteria associated with the plurality of data blocks. The second criteria include the first criteria, and the first criteria characterize input/output (I/O) activity associated with the data block. The methods may also include identifying data stores based on the first criteria, the data stores being included in data storage tiers. The data stores are identified based on the I/O activity associated with the data block. The methods further include updating the first criteria stored in the index and associated with the data block.
1. A method comprising:
identifying a first plurality of criteria associated with a data block included in a plurality of data blocks, the data block being identified based on a blockmap and an index, the blockmap characterizing a file stored in a deduplication repository, the index identifying data store identifiers, reference count data, and a second plurality of criteria associated with the plurality of data blocks, the second plurality of criteria including the first plurality of criteria, the first plurality of criteria characterizing input/output (I/O) activity associated with the data block;
identifying at least one data store based on at least one of the first plurality of criteria, the at least one data store being included in a data storage tier of a plurality of data storage tiers, the at least one data store being identified based, at least in part, on the I/O activity associated with the data block; and
updating the first plurality of criteria stored in the index and associated with the data block.
2. The method of claim 1 , wherein the first plurality of criteria comprises a first data store identifier, second data store identifier, a reference count, an access frequency metric, and a data access timestamp.
3. The method of claim 2 , wherein the access frequency metric characterizes a frequency at which the data block is accessed.
4. The method of claim 2 further comprising:
implementing at least one data transfer operation on the data block, the data transfer operation being identified based on the first plurality of criteria,
wherein the at least one data store is a first data store of a plurality of data stores, and wherein the plurality of data stores further comprises a second data store.
5. The method of claim 4 , wherein the data transfer operation is a data replication operation from the first data store included in a first data storage tier to the second data store included in a data second storage tier.
6. The method of claim 4 , wherein the data transfer operation is a migrate operation from the first data store included in a data storage tier to a second data store included in a second data storage tier.
7. The method of claim 4 , wherein the plurality of data stores further comprises a third data store included in a third data storage tier and a fourth data store included in a fourth data storage tier.
8. The method of claim 7 , wherein the first data storage tier has the lowest access latency, wherein the second data storage tier has the second lowest access latency, wherein the third data storage tier has the third lowest access latency, and wherein the fourth data storage tier has the highest access latency.
9. The method of claim 1 , wherein the at least one data store and an associated data storage tier is identified based on the index.
10. The method of claim 9 further comprising:
generating the index based on a plurality of I/O patterns associated with a plurality of data blocks included in the blockmap of the deduplication repository,
wherein the I/O patterns characterize data storage and access requests associated with the plurality of data blocks.
11. A system comprising:
a local machine;
a networked storage device comprising one or more processors configured to:
identify a first plurality of criteria associated with a data block included in a plurality of data blocks, the data block being identified based on a blockmap and an index, the blockmap characterizing a file stored in a deduplication repository, the index identifying data store identifiers, reference count data, and a second plurality of criteria associated with the plurality of data blocks, the second plurality of criteria including the first plurality of criteria, the first plurality of criteria characterizing input/output (I/O) activity associated with the data block;
identify at least one data store based on at least one of the first plurality of criteria, the at least one data store being included in a data storage tier of a plurality of data storage tiers, the at least one data store being identified based, at least in part, on the I/O activity associated with the data block; and
update the first plurality of criteria stored in the index and associated with the data block.
12. The system of claim 11 , wherein the first plurality of criteria comprises a first data store identifier, second data store identifier, a reference count, an access frequency metric, and a data access timestamp, and wherein the access frequency metric characterizes a frequency at which the data block is accessed.
13. The system of claim 12 , wherein the one or more processors are further configured to:
implement at least one data transfer operation on the data block, the data transfer operation being identified based on the plurality of criteria,
wherein the at least one data store is a first data store of a plurality of data stores, and wherein the plurality of data stores further comprises a second data store.
14. The system of claim 11 , wherein the at least one data store is identified based on the index.
15. The system of claim 14 , wherein the one or more processors are further configured to:
generate the index based on a plurality of I/O patterns associated with a plurality of data blocks included in the blockmap of the deduplication repository,
wherein the I/O patterns characterize data storage and access requests associated with the plurality of data blocks.
16. A device comprising:
a communications interface configured to be communicatively coupled with a local machine; and
one or more processors configured to:
identify a first plurality of criteria associated with a data block included in a plurality of data blocks, the data block being identified based on a blockmap and an index, the blockmap characterizing a file stored in a deduplication repository, the index identifying data store identifiers, reference count data, and a second plurality of criteria associated with the plurality of data blocks, the second plurality of criteria including the first plurality of criteria, the first plurality of criteria characterizing input/output (I/O) activity associated with the data block;
identify at least one data store based on at least one of the first plurality of criteria, the at least one data store being included in a data storage tier of a plurality of data storage tiers, the at least one data store being identified based, at least in part, on the I/O activity associated with the data block; and
update the first plurality of criteria stored in the index and associated with the data block.
17. The device of claim 16 , wherein the first plurality of criteria comprises a first data store identifier, second data store identifier, a reference count, an access frequency metric, and a data access timestamp, and wherein the access frequency metric characterizes a frequency at which the data block is accessed.
18. The device of claim 17 , wherein the one or more processors are further configured to:
implement at least one data transfer operation on the data block, the data transfer operation being identified based on the plurality of criteria,
wherein the at least one data store is a first data store of a plurality of data stores, and wherein the plurality of data stores further comprises a second data store.
19. The device of claim 16 , wherein the at least one data store is identified based on the index.
20. The device of claim 19 , wherein the one or more processors are further configured to:
generate the index based on a plurality of I/O patterns associated with a plurality of data blocks included in the blockmap of the deduplication repository,
wherein the I/O patterns characterize data storage and access requests associated with the plurality of data blocks.