Data storage in a cloud-based storage system
Data resiliency in a cloud-based storage system, including: receiving, for storage within a first tier of cloud storage of the cloud-based storage system, one or more segments of data; generating, for each of one or more shards of data of the one or more segments of data, self-describing information for recoverability of the one or more shards of data; and storing, within a second tier of cloud storage of the cloud-based storage system, both the one or more shards of data and the generated self-describing information for recoverability of the one or more shards of data.
1. A method comprising:
receiving data for storage within a first tier of cloud storage of a cloud-based storage system;
generating self-describing information for recoverability of the data before storing the data in a second tier of cloud storage of the cloud-based storage system;
storing the self-describing information on a number of storage elements that is sufficient for recovery of the data; and
storing, within the second tier, both the data and the self-describing information for recoverability of the data.
2. The method of claim 1 , further comprising removing the self-describing information for recoverability from the second tier of cloud storage of the cloud-based storage system.
3. The method of claim 1 , wherein the self-describing information for recoverability of the data includes information about distribution of the data in the cloud-based storage system and redundancy information.
4. The method of claim 1 , wherein the received data are included within one or more blocks.
5. The method of claim 1 , wherein the first tier of cloud storage includes a plurality of cloud computing instances.
6. The method of claim 1 , wherein the first tier of cloud storage is block-level storage.
7. The method of claim 1 , wherein the second tier of cloud storage includes an object storage.
8. The method of claim 1 , wherein the first tier of cloud storage includes a level of storage granularity that is block-level, and wherein storing the data includes transferring the data from block-level storage in the first tier of cloud storage to object storage in the second tier of cloud storage.
9. The method of claim 1 , wherein the first tier of cloud storage of the cloud-based storage system stores a subset of a volume of data.
10. The method of claim 1 , wherein the data are part of a dataset, and wherein the cloud-based storage system synchronously replicates the dataset with one or more other storage systems.
11. A cloud-based storage system within a cloud computing environment, the cloud-based storage system including:
one or more cloud computing instances providing a first tier of cloud storage;
a second tier of cloud storage; and
one or more storage controller applications, each storage controller application executing in a cloud computing instance, wherein the one or more storage controller applications are configured to:
receive data for storage within a first tier of cloud storage of a cloud-based storage system;
generate self-describing information for recoverability of the data before storing the data in a second tier of cloud storage of the cloud-based storage system;
storing the self-describing information on a number of storage elements that is sufficient for recovery of the data; and
store, within the second tier, both the data and the self-describing information for recoverability of the data.
12. The cloud-based storage system of claim 11 , wherein the one or more storage controller applications are further configured to remove the self-describing information for recoverability from the second tier of cloud storage of the cloud-based storage system.
13. The cloud-based storage system of claim 11 , wherein the self-describing information for recoverability of the data includes information about distribution of the data in the cloud-based storage system and redundancy information.
14. The cloud-based storage system of claim 11 , wherein the received data are included within one or more blocks.
15. The cloud-based storage system of claim 11 , wherein the first tier of cloud storage is block-level storage.
16. The cloud-based storage system of claim 11 , wherein the second tier of cloud storage includes an object storage.
17. The cloud-based storage system of claim 11 , wherein the first tier of cloud storage includes a level of storage granularity that is block-level, and wherein storing the data includes transferring the data from block-level storage in the first tier of cloud storage to object storage in the second tier of cloud storage.
18. The cloud-based storage system of claim 11 , wherein the first tier of cloud storage of the cloud-based storage system stores a subset of a volume of data.
19. The cloud-based storage system of claim 11 , wherein the data are part of a dataset, and wherein the cloud-based storage system synchronously replicates the dataset with one or more other storage systems.
20. The cloud-based storage system of claim 19 , wherein the one or more other storage systems include a hardware-based storage system.