Massively scalable object storage for storing object replicas
View Patent ↗An example method for storing data includes providing a plurality of physical storage pools, each storage pool including a plurality of storage nodes coupled to a network. The method also includes mapping a partition of a plurality of partitions to a set of physical storage pools, where each physical storage pool of the set of physical storage pools is located in a different availability zone, and the storage nodes within an availability zone are subject to a correlated loss of access to stored data. The method further includes receiving a data management request over the network, the data management request being associated with a data object. The method also includes identifying a first partition of the plurality of partitions corresponding to the received data management request and manipulating the data object in the physical storage pools mapped to the first partition in accordance with the data management request.
1. A method of building one or more rings for maintaining data in a file storage system, comprising:
determining a number of partitions for each storage pool in the file storage system, wherein the number of partitions is based on a storage capacity of the respective storage pool;
placing the storage pools in a sorting order based on the number of partitions;
assigning partitions to storage pools based on the sorting order, but subject to a zone restriction that the storage pool to which a partition is being assigned is not in a same zone as any other storage pool that includes a partition replica for the partition; and
maintaining the sorting order as partitions are assigned to storage pools in the file storage system, wherein maintaining the sorting order includes decrementing the respective number of partitions for the respective storage pool to which the respective partition has been assigned.
2. The method of claim 1 , wherein the number of partitions is based on a weight of the respective storage pool.
3. The method of claim 1 , further including:
determining a partition power for the respective storage pool; and
calculating 2 to the partition power, wherein the number of partitions is based on a result of the calculating.
4. The method of claim 3 , further including:
assigning N partitions to the respective storage pool, wherein N is a result of the calculating 2 to the partition power.
5. The method of claim 1 , wherein the placing the storage pools in a sorting order including placing the storage pools in an order from the highest number of partitions to the lowest number of partitions.
6. The method of claim 5 , further including:
maintaining the sorting order as partitions are assigned to storage pools in the file storage system.
7. The method of claim 1 , wherein the assigning partitions to storage pools includes assigning partitions to a storage pool with the highest number of partitions.
8. The method of claim 1 , wherein the maintaining further includes moving the respective storage pool to a lower position in the sorting order.
9. The method of claim 1 , wherein each partition is assigned multiple storage pools in different zones.
10. The method of claim 1 , further including:
receiving an object;
associating the object with a set of partitions assigned to storage pools in different zones; and
storing a replica of the object in each partition of the set of partitions.
11. The method of claim 1 , wherein a ring is an object ring.
12. The method of claim 1 , wherein a ring is a container ring.
13. The method of claim 1 , wherein a ring is an account ring.
14. A non-transitory computer readable medium containing executable instructions, which when executed on one or more processors:
determines a number of partitions for each storage pool in the file storage system wherein the number of partitions is based on a storage capacity of the respective storage pool;
places the storage pools in a sorting order based on the number of partitions;
assigns partitions to storage pools based on the sorting order, but subject to a zone restriction that the storage pool to which a partition is being assigned is not in a same zone as any other storage pool that includes a partition replica for the partition; and
maintains the sorting order as partitions are assigned to storage pools in the file storage system, wherein maintaining the sorting order includes decrementing the respective number of partitions for the respective storage pool to which the respective partition has been assigned.
15. The computer readable medium of claim 14 , wherein the number of partitions is based on a weight of the respective storage pool.
16. The computer readable medium of claim 14 , the executable instructions further including:
determining a partition power for the respective storage pool;
calculating 2 to the partition power; and
assigning N partitions to the respective storage pool, wherein N is a result of the calculating 2 to the partition power.