Positional indexing for a tiered data storage system
The system and methods disclosed herein relate to an improvement in automated data tiering technology. The systems and methods disclosed herein enhance database storage performance characteristics in myriad ways. First, the speed with which data can be relocated from one tier to another in a tiered data storage system is increased by reducing the number of sort cycles required to perform data relocation. In addition, data relocation among the tiers is performed on the backend by an offload engine, which results in uninterrupted access to read/write commands within the data storage system from a user's perspective on the frontend. Third, users are able to adjust the percentages of hot or cold data that are relocated within the database without having to alter the service level agreements. In this way, users can make spontaneous changes to performance characteristics related to the promotion and demotion of data stored within a tiered data storage system.
1. A method for sorting data in a data storage system comprising a storage subsystem, wherein the storage subsystem comprises a processing unit, a storage subsystem memory, and a plurality of storage tiers having differing performance characteristics, and an offload engine, wherein the offload engine comprises a processing unit and an offload memory, the method comprising:
a. receiving a first metric associated with a desired percentage of data stored in a sorted array to be categorized as hot data;
b. receiving a second metric associated with a desired percentage of data stored in the sorted array to be categorized as cold data;
c. copying the sorted array from the storage subsystem to the offload engine;
d. separating the sorted array using the offload engine, into first and second partitioned storage sectors, thereby creating a second sorted array, wherein the first partitioned storage sector in the second array contains hot data sorted in descending order, and the second partitioned storage sector in the second array contains cold data sorted in ascending order; and
e. writing the second sorted array into the storage subsystem memory.
2. The method of claim 1 wherein the first metric and the second metric are received in the offload engine.
3. The method of claim 1 wherein the first metric and the second metric are received in the storage subsystem.
4. The method of claim 1 wherein the storage subsystem memory further comprises a database.
5. The method of claim 1 wherein the offload engine is external to data storage system.
6. A system comprising:
a. a storage subsystem including a processing unit, a storage subsystem memory, and a plurality of storage tiers having different performance characteristics;
b. an offload engine including a processing unit and an offload memory communicatively coupled to the storage subsystem;
c. computer-executable logic, wherein the computer-executable logic is configured for the execution of:
i. receiving a first metric associated with a desired percentage of data stored in the offload engine to be categorized as hot data;
ii. receiving a second metric associated with a desired percentage of data stored in the offload engine to be categorized as cold data;
iii. copying a sorted array from the storage subsystem to the offload engine;
iv. separating the sorted array using the offload engine, into first and second partitioned storage sectors, thereby creating a second sorted array, wherein the first partitioned storage sector in the second array contains a plurality of hot data sorted in descending order, and the second partitioned storage sector in the second array contains a plurality of cold data sorted in ascending order; and
v. writing the second sorted array into the storage subsystem memory.
7. The system of claim 6 wherein the first metric and the second metric are received in the offload engine.
8. The system of claim 6 wherein the first metric and the second metric are received in the storage subsystem.
9. The system of claim 6 wherein the storage subsystem memory further comprises a database.
10. The system of claim 6 wherein the offload engine is external to the data storage system.
11. The system of claim 6 wherein the computer executable logic is operating on the offload engine.
12. A computer program product for sorting data in a data storage system comprising a storage subsystem, wherein the storage subsystem comprises a processing unit, a storage subsystem memory, and a plurality of storage tiers having differing performance characteristics, and an offload engine, wherein the offload engine comprises a processing unit and an offload memory, the computer program product comprising a non-transitory computer readable medium encoded with computer-executable program code the code configured to enable the execution of:
a. receiving a first metric associated with a desired percentage of data stored in a sorted array to be categorized as hot data;
b. receiving a second metric associated with a desired percentage of data stored in the sorted array to be categorized as cold data;
c. copying the sorted array from the storage subsystem to the offload engine;
d. separating the sorted array using the offload engine, into first and second partitioned storage sectors, thereby creating a second sorted array, wherein the first partitioned storage sector in the second array contains hot data sorted in descending order, and the second partitioned storage sector contains in the second array cold data sorted in ascending order; and
e. writing the second sorted array into the storage subsystem memory or a database located within the physical storage pool.
13. The computer program product of claim 12 wherein the first metric and the second metric are received in the offload engine.
14. The computer program product of claim 12 wherein the first metric and the second metric are received in the storage subsystem.
15. The computer program product of claim 12 wherein the storage subsystem memory further comprises a database.
16. The computer program product of claim 12 wherein the offload engine is external to the data storage system.