Data storage system including unique block pool manager and applications in tiered storage
View Patent ↗A data storage system ( 10 ) that receives a data set from a software module ( 18 A-D) includes a first tier storage device ( 240 ), a second tier storage device ( 242 ), a redundancy reducer ( 21 ) and a migration engine ( 28 ). The first tier storage device ( 240 ) has a first effective storage capacity and the second tier storage device ( 242 ) can have a second effective storage capacity that is greater than the first effective storage capacity. The redundancy reducer ( 21 ) subdivides the data set into a plurality of data blocks ( 20 ) and reduces the redundancy of the data blocks ( 20 ). The migration engine ( 28 ) moves one or more of the data blocks ( 20 ) between the first tier storage device ( 240 ) and the second tier storage device ( 242 ) based on a migration parameter of the data block ( 20 ). The first tier storage device ( 240 ) can store data in a random access manner and the second tier storage device ( 242 ) can store data in a random or sequential access manner. The first tier storage device ( 240 ) has a first I/O bandwidth, and the second tier storage device ( 242 ) can have a second I/O bandwidth that is less than the first I/O bandwidth. The first tier storage device ( 240 ) has a first access time to data, and the second tier storage device ( 242 ) can have a second access time to data that is lengthier than the first access time to data. The data storage system ( 10 ) can also include a third tier storage device ( 238 ) used for retired data blocks ( 20 ).
1. A data storage system that receives a data set from a software module, the data storage system comprising:
a first tier storage device having a first effective storage capacity;
a second tier storage device having a second effective storage capacity that is greater than the first effective storage capacity;
a redundancy reducer that subdivides the data set into a plurality of data blocks and reduces the redundancy of the data blocks; and
a migration engine that moves one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block, wherein the data set includes (i) a first data pool having a plurality of first data blocks, and (ii) a separately-stored second data pool having a plurality of second data blocks, and wherein the migration engine is adapted to merge the first data pool with the second data pool.
2. The data storage system of claim 1 wherein the redundancy reducer includes a hash engine.
3. The data storage system of claim 1 wherein the redundancy reducer includes a lookup engine.
4. The data storage system of claim 1 wherein the migration parameter includes a reference count.
5. The data storage system of claim 1 wherein the migration parameter includes a last access time.
6. The data storage system of claim 1 wherein the migration parameter includes an access rate.
7. The data storage system of claim 1 wherein the migration parameter includes a user-defined access policy.
8. The data storage system of claim 1 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks by the migration engine.
9. The data storage system of claim 1 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set prior to reduction of the redundancy of the data blocks by the redundancy reducer.
10. The data storage system of claim 9 wherein the second tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.
11. The data storage system of claim 10 further comprising a third tier storage device having a third effective storage capacity that is greater than the second effective storage capacity.
12. The data storage system of claim 11 wherein the migration engine moves one of the data blocks between the second tier storage device and the third tier storage device based at least partially on a migration parameter of the data block.
13. The data storage system of claim 11 wherein the third tier storage device includes a tape library.
14. The data storage system of claim 11 wherein the third tier storage device includes a massive array of idle disks.
15. The data storage system of claim 9 wherein the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.
16. The data storage system of claim 1 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.
17. The data storage system of claim 16 wherein the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.
18. The data storage system of claim 1 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.
19. The data storage system of claim 1 wherein the first tier storage device stores data in a random access manner, and the second tier storage device stores data in a sequential access manner.
20. The data storage system of claim 1 wherein the first tier storage device stores data in a random access manner, and the second tier storage device stores data in a random access manner.
21. The data storage system of claim 1 wherein the first tier storage device has a first I/O bandwidth, and the second tier storage device has a second I/O bandwidth that is less than the first I/O bandwidth.
22. The data storage system of claim 1 wherein the first tier storage device has a first access time to data, and the second tier storage device has a second access time to data that is lengthier than the first access time to data.
23. A data storage system that receives a data set from a software module, the data storage system comprising:
a first tier storage device that stores data in a random access manner;
a second tier storage device that stores data in a sequential access manner;
a redundancy reducer, wherein the data set includes a first data pool and a separately-stored second data pool, the redundancy reducer subdividing the first data pool into a plurality of first data blocks and reducing the redundancy of the first data blocks, and the redundancy reducer subdividing the second data pool into a plurality of second data blocks and reducing the redundancy of the second data blocks; and
a migration engine that moves one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block, the migration engine being adapted to merge the first data pool with the second data pool.
24. The data storage system of claim 23 wherein the redundancy reducer includes a hash engine and a lookup engine.
25. The data storage system of claim 23 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.
26. The data storage system of claim 23 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks by the migration engine.
27. The data storage system of claim 23 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer, and the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.
28. The data storage system of claim 23 wherein the second tier storage device includes one of a tape library and a massive array of idle disks.
29. The data storage system of claim 23 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.
30. The data storage system of claim 23 wherein the first tier storage device has a first I/O bandwidth, and the second tier storage device has a second I/O bandwidth that is less than the first I/O bandwidth.
31. The data storage system of claim 23 wherein the first tier storage device has a first access time to data, and the second tier storage device has a second access time to data that is lengthier than the first access time to data.
32. A data storage system that receives a data set from a software module, the data storage system comprising:
a first tier storage device having a first I/O bandwidth;
a second tier storage device having a second I/O bandwidth that is less than the first I/O bandwidth;
a redundancy reducer, wherein the data set includes a first data pool and a separately-stored second data pool, the redundancy reducer subdividing the first data pool into a plurality of first data blocks and reducing the redundancy of the first data blocks; and
a migration engine that moves one of the first data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the first data block, the migration engine being adapted to merge the first data pool with the second data pool.
33. The data storage system of claim 32 wherein the redundancy reducer includes at least one of a hash engine and a lookup engine.
34. The data storage system of claim 32 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.
35. The data storage system of claim 32 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the first data blocks by the migration engine.
36. The data storage system of claim 32 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set prior to reduction of the redundancy of the first data blocks by the redundancy reducer, and the second tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the first data blocks by the redundancy reducer.
37. The data storage system of claim 36 further comprising a third tier storage device having a third I/O bandwidth that is less than the second I/O bandwidth.
38. The data storage system of claim 37 wherein the migration engine moves one of the first data blocks between the second tier storage device and the third tier storage device based at least partially on a migration parameter of the first data block.
39. The data storage system of claim 38 wherein the third tier storage device includes one of a tape library and a massive array of idle disks.
40. The data storage system of claim 32 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the first data blocks by the redundancy reducer, and the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the first data blocks by the redundancy reducer.
41. The data storage system of claim 32 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.
42. The data storage system of claim 32 wherein the first tier storage device has a first access time to data, and the second tier storage device has a second access time to data that is lengthier than the first access time to data.
43. A data storage system that receives a data set from a software module, the data storage system comprising:
a first tier storage device having a first access time to data;
a second tier storage device having a second access time to data that is lengthier than the first access time to data;
a redundancy reducer, wherein the data set includes a first data pool and a separately-stored second data pool, the redundancy reducer subdividing the first data pool into a plurality of first data blocks and reducing the redundancy of the first data blocks; and
a migration engine that moves one of the first data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the first data block, the migration engine being adapted to merge the first data pool with the second data pool.
44. The data storage system of claim 43 wherein the redundancy reducer includes at least one of a hash engine and a lookup engine.
45. The data storage system of claim 43 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.
46. The data storage system of claim 43 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the first data blocks between the storage devices by the migration engine.
47. The data storage system of claim 43 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set prior to reduction of the redundancy of the first data blocks by the redundancy reducer, and the second tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the first data blocks by the redundancy reducer.
48. The data storage system of claim 47 further comprising a third tier storage device having a third access time to data that is lengthier than the second access time to data.
49. The data storage system of claim 48 wherein the migration engine moves one of the first data blocks between the second tier storage device and the third tier storage device based at least partially on a migration parameter of the first data block.
50. The data storage system of claim 49 wherein the third tier storage device includes one of a tape library and a massive array of idle disks.
51. The data storage system of claim 43 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the first data blocks by the redundancy reducer, and the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the first data blocks by the redundancy reducer.
52. The data storage system of claim 43 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.
53. A data storage system that receives a data set from a software module, the data storage system comprising:
a first tier storage device having a first effective storage capacity;
a second tier storage device having a second effective storage capacity that is greater than the first effective storage capacity; and
a migration engine, wherein the data set includes (i) a first data pool having a plurality of first data blocks, and (ii) a separately-stored second data pool having a plurality of second data blocks, the migration engine moving one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block, and the migration engine being adapted to merge the first data pool with the second data pool.
54. The data storage system of claim 53 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.
55. The data storage system of claim 53 further comprising a redundancy reducer that reduces the redundancy of the data blocks.