Fault tolerant distributed storage for cloud computing
View Patent ↗Data sets and blocks are stored in a set of independent, functionally equivalent chunks. These chunks are placed on different elements of a distributed network to achieve pre-defined level of fault tolerance. Terms of fault tolerance are defined in terms of amount of unavailable sites in the network allowing receipt and access to the data block. Maximal and minimal number of chunks available are variable method parameters. The minimal amount of data chunks K needed to restore a data block is defined. The size of each chunk is approximately 1/K of the original block size. The maximal amounts of chunks are defined during distribution operation and depend upon a requested fault tolerance level. Redundancy in data storage is minimized and varies dynamically by changing the total amount of chunks available. Significant increase in data transfer rate is possible because all block chunks could be transferred in parallel and independently.
1. A method for fault tolerant data storage of a data object in a cloud computing system formed by a plurality of arbitrarily connected data servers, the method comprising:
defining a size of data block into which the data object will be divided;
receiving, from a user, a fault tolerance level;
dividing the data object into a plurality of data blocks;
based on the tolerance level, defining a minimal number K out of N data chunks needed to restore one data block;
disassembling each of the data blocks into at least L different data chunks, wherein K≦L≦N and L depends on the fault tolerance level selected by the user; and
distributing the data chunks to the plurality of arbitrarily connected data servers.
2. The method of claim 1 , wherein all the data chunks are written to the storage elements placed on a single storage medium.
3. The method of claim 2 , wherein the storage elements are allocated throughout the storage medium so as maximize distribution of data related to different chunks of a single data block.
4. The method of claim 2 , wherein the storage elements are allocated separately by distance.
5. The method of claim 2 , further comprising:
defining geometrical areas of the data storage medium for placing storage elements corresponding to different chunks of one block; and
writing the chunks to storage elements located within the geometrical areas.
6. The method of claim 1 , wherein the server of the cloud computing system is used as a controller for performing distribution of chunks and operations with chunks.
7. The method of claim 1 , wherein the controller requests only K chunks from the data servers.
8. The method of claim 7 , wherein, if all the requested K chunks are not available, the controller requests additional chunks from the data servers.
9. The method of claim 1 , wherein devices that act as data servers do not require authentication.
10. The method of claim 1 , wherein the fault tolerance level is changeable by the user after the data blocks are distributed to the data servers.
11. The method of claim 1 , wherein the availability is implemented by providing simultaneous availability of M data chunks.
12. The method of claim 1 , wherein a fault tolerance level is changeable by the user after the data blocks are distributed to the data servers.
13. The method of claim 12 , wherein the fault tolerance level is adjustable based on a number of data server available at any given time and a number of unavailable servers.
14. The method of claim 1 , wherein an availability level is changeable by the user after the data blocks are distributed to the data servers.
15. The method of claim 1 , wherein the chunks are placed so as to satisfy a required level of availability, fault tolerance level and/or bandwidth.
16. The method of claim 1 , wherein the number L depends on any of a desired fault tolerance level, a network bandwidth, available system resources, desired availability and workload.
17. A system for fault tolerant data storage of a data object, the system comprising:
a plurality of data servers forming the cloud computing system;
means for defining a size of data block into which the data object will be divided;
a fault tolerance level received from a user;
means for dividing the data object into a plurality of data blocks;
based on the fault tolerance level, means for defining a minimal number K out of N data chunks needed to restore one data block;
a controller that disassembles each of the data blocks into at least L different data chunks and distributes the data chunks to the plurality of connected data servers,
wherein K≦L≦N and L depends on the fault tolerance level selected by the user; and
wherein the fault tolerance level is adjustable based on a number of data server available at any given time and a number of unavailable servers.
18. A method for providing high availability of access to a data object in a cloud computing system formed by a plurality of arbitrarily connected data servers, the method comprising:
defining a size of data block into which the data object will be divided;
receiving, from a user, a desired availability level;
dividing the data object into a plurality of data blocks;
based on the desired availability level, defining a minimal number K out of N data chunks needed to restore one data block;
disassembling each of the data blocks into at least L different data chunks, wherein K≦L≦N and L depends on the desired availability level selected by the user; and
distributing the data chunks to the plurality of arbitrarily connected data servers.
19. A system for providing high availability of access to a data object in a cloud computing system formed by a plurality of data servers, the system comprising:
a plurality of data servers forming the cloud computing system;
means for defining a size of data block into which the data object will be divided;
an availability level received from a user;
means for dividing the data object into a plurality of data blocks;
based on the availability level, means for defining a minimal number K out of N data chunks needed to restore one data block;
a controller that disassembles each of the data blocks into at least L different data chunks and distributes the data chunks to the plurality of connected data servers,
wherein K≦L≦N and L depends on the fault tolerance level selected by the user; and
wherein the availability level is adjustable based on a predictable number of data servers available at any given time.