Consistent recovery of a dataset
Servicing I/O operations in a cloud-based storage system, including: receiving, by the cloud-based storage system, a request to write data to the cloud-based storage system; storing, in solid-state storage of the cloud-based storage system, the data; storing, in object storage of the cloud-based storage system, the data; detecting that at least some portion of the solid-state storage of the cloud-based storage system has become unavailable; identifying data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; retrieving, from object storage of the cloud-based storage system, the data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; and storing, in solid-state storage of the cloud-based storage system, the retrieved data.
1. A method comprising:
storing tracking metadata for at least some of multiple portions of data stored within a first tier of cloud storage, wherein the tracking metadata is stored outside of the first tier of cloud storage and the multiple portions of data correspond to multiple storage operations; and
determining, after a data loss event among one or more storage repositories of the first tier of cloud storage and based on the tracking metadata, a consistent set of data among functioning storage repositories of the first tier of cloud storage, wherein each of the multiple storage operations corresponds to an identifier within a single time period identifier, and wherein a storage operation is committed based on storage of the tracking metadata.
2. The method of claim 1 , wherein the first tier of cloud storage includes a plurality of cloud computing instances that store the multiple portions of data.
3. The method of claim 2 , wherein the cloud computing instances provide block-level data storage.
4. The method of claim 2 , wherein the data loss event is a failure of one or more of the cloud computing instances.
5. The method of claim 4 , wherein the functioning storage repositories include one or more cloud computing instances created to replace the failed one or more of the cloud computing instances.
6. The method of claim 1 , wherein a second tier of cloud storage includes a cloud-based object storage.
7. The method of claim 1 , wherein the first tier of cloud storage has one or more better performance characteristics than a second tier of cloud storage.
8. The method of claim 1 further comprising receiving, into one or more storage repositories of the first tier of cloud storage, multiple portions of data corresponding to multiple storage operations.
9. The method of claim 1 , further comprising:
responsive to the data loss event, and based on tracking metadata that is committed, invalidating one or more of the multiple storage operations.
10. The method of claim 1 , further comprising:
notifying a garbage collection process to not garbage collect tracking metadata for one or more storage operations.
11. A cloud-based storage system including:
one or more cloud computing instances providing a first tier of cloud storage;
a second tier of cloud storage; and
one or more storage controller applications, each storage controller application executing in a cloud computing instance, wherein the one or more storage controllers are configured to:
store tracking metadata for at least some of multiple portions of data stored within the first tier of cloud storage, wherein the tracking metadata is stored outside of the first tier of cloud storage and the multiple portions of data correspond to multiple storage operations; and
determine, after a data loss event among one or more storage repositories of the first tier of cloud storage and based on the tracking metadata, a consistent set of data among functioning storage repositories of the first tier of cloud storage, wherein each of the multiple storage operations corresponds to an identifier within a single time period identifier, and wherein a storage operation is committed based on storage of the tracking metadata.
12. The cloud-based storage system of claim 11 , wherein the first tier of cloud storage of the cloud-based storage system includes a plurality of cloud computing instances that store the multiple portions of data or storage operations.
13. The cloud-based storage system of claim 12 , wherein the cloud computing instances provide block-level data storage.
14. The cloud-based storage system of claim 12 , wherein storing, in block storage of the cloud-based storage system, the data further comprises storing, in local storage of one or more cloud computing instances, the data.
15. The cloud-based storage system of claim 12 , wherein the data loss event is a failure of one or more of the cloud computing instances.
16. The cloud-based storage system of claim 15 , wherein the functioning storage repositories include one or more cloud computing instances created to replace the failed one or more of the cloud computing instances.
17. The cloud-based storage system of claim 11 , wherein the second tier of cloud storage of the cloud-based storage system includes a cloud-based object storage.
18. The cloud-based storage system of claim 11 , wherein the first tier of cloud storage has one or more better performance characteristics than the second tier of cloud storage.
19. The cloud-based storage system of claim 11 wherein the one or more storage controllers are further configured to receive, into one or more storage repositories of the first tier of cloud storage of the cloud-based storage system, multiple portions of data corresponding to multiple storage operations.