Expanding a distributed storage system
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient stripes is distributed across the plurality of storage devices such that each of the plurality of failure resilient stripes spans a plurality of the storage devices. The plurality of computing devices maintains each failure resilient stripe using a bucket. These buckets are operably split and transferred between the computing devices to balance available computing power and storage access.
1. A system comprising:
a first computing device comprising a first backend; and
a plurality of storage devices, wherein:
the first backend comprises one or more buckets,
each bucket of the one or more buckets is operable to build a plurality of failure resilient stripes such that each of the plurality of failure resilient stripes comprises a plurality of storage blocks,
each storage block of the plurality of storage blocks in a failure resilient stripe is located in a different storage device of the plurality of storage devices, and
if the first backend comprises at least two buckets, the first computing device is operable to transfer a bucket of the at least two buckets to a second backend on a second computing device.
2. The system of claim 1 , wherein the first computing device is operable to split a bucket on the first backend into a first split bucket and a second split bucket.
3. The system of claim 2 , wherein a plurality of failure resilient stripes associated with the bucket on the first backend is distributed among the first split bucket and the second split bucket.
4. The system of claim 2 , wherein the first computing device is operable to split the first split bucket into a third split bucket and a fourth split bucket.
5. The system of claim 1 , wherein each bucket on the first backend is operable to build a new failure resilient stripe according to an available capacity of the plurality of storage devices, and wherein a most recently added storage device of the plurality of storage devices is selected for the new failure resilient stripe.
6. A system comprising:
a first computing device comprising a first backend; and
a plurality of storage devices, wherein:
the first backend comprises one or more buckets,
each bucket of the one or more buckets is operable to build a failure resilient stripe such that the failure resilient stripe comprises a plurality of storage blocks,
each storage block of the plurality of storage blocks in the failure resilient stripe is located in a different storage device of the plurality of storage devices,
the different storage devices of the plurality of storage devices are selected according to an available capacity of each storage device of the plurality of storage devices, and
a most recently added storage device of the plurality of storage devices is selected prior to an occupied storage device.
7. The system of claim 6 , wherein if the first backend comprises at least two buckets, the first computing device is operable to transfer a bucket of the at least two buckets to a second backend on a second computing device.
8. The system of claim 6 , wherein the first computing device is operable to split a bucket on the first backend into a first split bucket and a second split bucket.
9. The system of claim 8 , wherein a plurality of failure resilient stripes associated with the bucket on the first backend is distributed among the first split bucket and the second split bucket.
10. The system of claim 8 , wherein the first computing device is operable to split the first split bucket into a third split bucket and a fourth split bucket.
11. A method comprising:
building a plurality of failure resilient stripes, using a bucket of the one or more buckets on a computing device backend, such that each of the plurality of failure resilient stripes comprises a plurality of storage blocks, wherein each storage block of the plurality of storage blocks in a failure resilient stripe is located in a different storage device of a plurality of storage devices; and
if the computing device backend comprises at least two buckets, transferring a bucket of the at least two buckets to a second backend on a second computing device.
12. The method of claim 11 , wherein the method comprises splitting a bucket on the first backend into a first split bucket and a second split bucket.
13. The method of claim 12 , wherein the method comprises distributing a plurality of failure resilient stripes associated with the bucket on the first backend among the first split bucket and the second split bucket.
14. The method of claim 12 , wherein the method comprises splitting the first split bucket into a third first split bucket and a fourth split bucket.
15. The method of claim 11 , wherein the method comprises building a new failure resilient stripe according to an available capacity of the plurality of storage devices, wherein a most recently added storage device of the plurality of storage devices is selected for the new failure resilient stripe.
16. A method comprising:
selecting different storage devices of a plurality of storage devices according to an available capacity of each storage device of the plurality of storage devices, wherein a most recently added storage device of the plurality of storage devices is selected prior to an occupied storage device;
building a failure resilient stripe, using a bucket of a first computing device, such that the failure resilient stripe comprises a plurality of storage blocks, wherein each storage block of the plurality of storage blocks in the failure resilient stripe is located in the different storage devices.
17. The method of claim 16 , wherein the method comprises transferring the bucket to a second computing device if the first computing device comprises at least two buckets.
18. The method of claim 16 , wherein the method comprises splitting the bucket on the first computing device into a first split bucket and a second split bucket.
19. The method of claim 18 , wherein the method comprises distributing a plurality of failure resilient stripes associated with the bucket on the first computing device among the first split bucket and the second split bucket.
20. The method of claim 18 , wherein the method comprises splitting the first split bucket into a third split bucket and a fourth split bucket.