Adaptive storage block data distribution
View Patent ↗Systems and methods for providing an efficient partitioned resource server. In one embodiment, the partitioned resource server comprises a plurality of individual servers, and the individual servers appear to be equivalent to a client. Each of the individual servers may include a routing table that includes a reference for each resource that is maintained on the partitioned resource server. Requests from a client are processed as a function of the routing table to route the request to the individual server that maintains or has control over the resource of interest.
1. A system for block level data storage, comprising:
a plurality of equivalent servers each being responsible for a portion of stored blocks and each of said plurality of equivalent servers having a respective data mover process capable of communicating directly with another respective data mover process on another server of said plurality of equivalent servers for coordinating movement of blocks between the plurality of equivalent servers;
each of said plurality of equivalent servers having a respective request monitor process communicating directly with another respective request monitor process on another server of said plurality of equivalent servers
each of said plurality of equivalent servers having a respective routing table process in each server and capable of communicating with another respective routing table process on each of the other servers of said plurality of equivalent servers, each process on each server for maintaining a coordinated list of equivalent servers that identifies the volume and server for each data block moved between the servers and data blocks associated therewith;
each of said respective request monitor process monitoring client requests to its respective server and communicating with other respective request monitor process to build a global request database;
said global database identifying patterns in the client requests, wherein the client requests access blocks of data;
each of said data mover process reallocating the blocks of data of the identified patterns from one server to another server of the plurality of equivalent servers based on operational characteristics of the plurality of equivalent servers and the identified patterns.
2. A system according to claim 1 , wherein each request monitor process identifies the patterns based on parameters selected from the group consisting of a number of clients accessing a data volume, a number of I/O requests, and a measure of available storage.
3. A system according to claim 1 , wherein
each request monitor process identifies the patterns based on whether a group of blocks are accessed together at a frequency above a hot block criterion.
4. A system according to claim 1 , wherein
the data mover process operates on data generated by the request monitor process reallocate a block of data to a selected server for improving performance.
5. A system according to claim 1 , wherein
the data mover process includes a data striping algorithm for striping selected data blocks across a plurality of servers.
6. A system according to claim 1 , wherein
the data mover process includes a transaction process for monitoring a transfer of data blocks between servers.
7. A system according to claim 6 , wherein
the transaction process includes an error log process for logging an error detected during the transfer of data blocks.
8. A system according to claim 1 , wherein
the request monitor monitors requests made of two or more data volumes.
9. A system according to claim 1 , wherein
the data mover process moves data blocks associated with two or more data volumes.
10. A process for providing block level data storage, comprising:
providing a plurality of equivalent servers each being responsible for a portion of stored data blocks and each monitoring client requests received by a respective equivalent server, each of said plurality of equivalent servers having a respective request monitor process and communicating directly with another respective request monitor process on another server of said plurality of equivalent servers;
moving data blocks between the servers; and
maintaining a respective routing table at each of said plurality of equivalent servers that identifies the volume and server for each data block moved between the servers and that communicates with other respective routing tables to generate a global routing table for tracking data blocks stored across different servers on the system;
each of said respective request monitor process monitoring client requests to its respective server and communicating with other respective request monitor process to build a global request database;
said global database identifying patterns in the client requests, wherein the client requests access blocks of data;
reallocating the blocks of data of the identified patterns from one server to another server of the plurality of equivalent servers based on operational characteristics of the plurality of equivalent servers and the identified patterns.
11. The process of claim 10 , wherein
the identified patterns are based on parameters selected from the group consisting of a number of clients accessing a data volume, a number of I/O requests, and a measure of available storage.
12. The process according to claim 10 , wherein
the identified patterns are based on whether a group of blocks are accessed together at a frequency above a hot block criterion.
13. The process according to claim 10 , wherein
data moving includes striping selected data blocks across a plurality of servers.
14. The process according to claim 10 , wherein
data moving includes monitoring a transfer of data blocks between servers.
15. The process according to claim 14 , further including
generating an error log for logging an error detected during the transfer of data blocks.
16. The process according to claim 10 , wherein
monitoring requests includes monitoring requests made of two or more data volumes.