Methods and systems of managing a distributed replica based storage
A method of managing a distributed storage space. The method comprises mapping a plurality of replica sets to a plurality of storage managing modules installed in a plurality of computing units, each of the plurality of storage managing modules manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit, allocating at least one time based credit to at least one of each storage managing module and the replica data, iteratively renewing the time based credit as long a failure of at least one of the storage managing module, and the at least one drive and the replica data is not detected plurality of storage managing.
1. A method of managing a distributed storage space, comprising;
mapping a plurality of replica sets to a plurality of storage managing modules installed in a plurality of computing units, each of the plurality of storage managing modules manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit; wherein the mapping maps blocks of volumes to a respective block address in a respective domain;
allocating at least one time based credit to at least one of each storage managing module, the at least one drive and the replica data;
iteratively renewing the time based credit as long a failure of at least one of the storage managing module, the at least one drive and the replica data is not detected; and
for a storage managing module that does not have a renewed credit, fencing the storage managing module by sending a declaration of failing to the other of the plurality of storage managing modules; wherein a fenced storage managing module stops servicing requests; reallocating the replica data to at least one other of the plurality of storage managing modules when the at least one time based credit is not renewed; wherein reallocating the replica data allows the replica data to be to be managed by other storage managing modules.
2. The method of claim 1 , further comprising instructing a respective the storage managing module to reject access of the at least one storage consumer application to the at least one replica.
3. The method of claim 1 , further comprising detecting a responsiveness of a respective the storage managing module and determining whether to reallocate the at least one replica to the storage managing module accordingly.
4. The method of claim 1 , wherein the plurality of replica sets are part of a volume stored in a plurality of drives managed by the plurality of storage managing modules.
5. The method of claim 1 , wherein each replica is divided to be stored in a plurality of volume allocation extents (VAEs) each define a range of consecutive addresses which comprise a physical segment in a virtual disk stored in the at least one drive.
6. The method of claim 1 , wherein each of a plurality of volume allocation extents (VAEs) of each of the plurality of replicas is divided to be stored in a plurality of physical segments each of another of a plurality of virtual disks which are managed by the plurality of storage managing modules so that access to different areas of each VAE is managed by different storage managing modules of the plurality of storage managing modules.
7. The method of claim 1 , wherein the plurality of computing units comprises a plurality of client terminals selected from a group consisting of desktops, laptops, tablets, and Smartphones .
8. The method of claim 1 , wherein each storage managing module manages a direct access of the at least one storage consumer application to a respective the at least one replica.
9. The method of claim 1 , wherein the mapping comprises allocating a first generation numerator to mapping element mapping the storage of the replica data, the reallocating comprises updating the first generation numerator; further comprising receiving a request to access the replica data with a second generation numerator and validating the replica data according to a match between the first generation numerator and the second generation numerator.
10. The method of claim 1 , further comprising performing a liveness check to the plurality of storage managing modules and performing the renewing based on an outcome of the liveness check.
11. The method of claim 1 , wherein the replica set is defined according to a member of a group consisting of the following protocols: Redundant Array of Independent Disks (RAID)-0 protocol, RAID-1, RAID-2, RAID-3, RAID-4, RAID-5 and RAID-6, RAID 10, RAID 20, RAID 30, RAID 40, RAID 50, RAID 60, RAID 01, RAID 02, RAID 03, RAID 04, RAID 05, and RAID 06 ; wherein the replica comprises at least one of a replica of data of a set of data elements and a parity of the set of data elements.
12. The method of claim 1 wherein the mapping is performed by one or more metadata servers.
13. The method of claim 12 wherein the one or more metadata servers manage mapping of a domain.
14. The method of claim 12 wherein the mapping includes a consumer mapping record mapping one or more volumes to one or more storage consumer module(s) including authorizing access to a volume based on the identity of the storage consumer module.
15. The method of claim 1 further comprising:
distributing information mapping from a replica address to a respective storage managing module to storage consumer modules; and
managing mapping information indicative of respective drive addresses by the storage managing modules.
16. A computer tangible non-transitory readable medium comprising computer executable logic enabling execution across one or more processors of:
mapping a plurality of replica sets to a plurality of storage managing modules installed in a plurality of computing units, each of the plurality of storage managing modules manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit; wherein the mapping maps blocks of volumes to a respective block address in a respective domain;
allocating at least one time based credit to at least one of each storage managing module, the at least one drive and the replica data;
iteratively renewing the time based credit as long a failure of at least one of the storage managing module, the at least one drive and the replica data is not detected; and
for a storage managing module that does not have a renewed credit, fencing the storage managing module by sending a declaration of failing to the other of the plurality of storage managing modules; wherein a fenced storage managing module stops servicing requests; reallocating the replica data to at least one other of the plurality of storage managing modules when the at least one time based credit is not renewed; wherein reallocating the replica data allows the replica data to be to be managed by other storage managing modules.
17. The computer tangible non-transitory readable medium of claim 16 the logic further enabling execution across one or more processors of:
distributing information mapping from a replica address to a respective storage managing module to storage consumer modules; and
managing mapping information indicative of respective drive addresses by the storage managing modules.
18. The computer tangible non-transitory readable medium of claim 16 the logic further enabling execution across one or more processors of performing a liveness check to the plurality of storage managing modules and performing the renewing based on an outcome of the liveness check.
19. The computer tangible non-transitory readable medium of claim 16 wherein the mapping is performed by one or more metadata servers.
20. A system of managing a distributed storage space, comprising;
mapping a plurality of storage managing modules installed in a plurality of computing units and manages the storage of a plurality of replica sets, each storage managing module manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit; wherein the mapping maps blocks of volumes to a respective block address in a respective domain; and
a central node which allocates at least one time based credit to at least one of each storage managing module and the replica data;
wherein the central node iteratively renews the time based credit as long a failure of at least one of the storage managing module, the at least one drive and the replica data is not detected; and
for a storage managing module that does not have a renewed credit, fencing the storage managing module by sending a declaration of failing to the other of the plurality of storage managing modules; wherein a fenced storage managing module stops servicing requests; further comprising, reallocating the replica data to at least one other of the plurality of storage managing modules when the at least one time based credit is not renewed; wherein reallocating the replica data allows the replica data to be to be managed by other storage managing modules; wherein the mapping maps blocks of volumes to a respective block address in a respective domain; reallocating the replica data to at least one other of the plurality of storage managing modules when the at least one time based credit is not renewed: wherein reallocating the replica data allows the replica data to be to be managed by other storage managing modules.