Maintaining consistent metadata across different storage systems
Symmetric storage using a cloud-based storage system, including: receiving, at a cloud-based storage system among storage systems synchronously replicating a dataset, an I/O operation directed to the dataset; determining, in dependence upon the I/O operation, a metadata update describing a mapping of segments of content to an address within a storage object, wherein the storage object includes the dataset; and synchronizing metadata on another storage system of the storage systems by sending the metadata update from the cloud-based storage system to the other storage system to update a metadata representation on the second storage system in accordance with the metadata update.
1 . A method comprising:
generating for a dataset on a cloud-based storage system, based on an I/O operation directed to the dataset, a metadata update that comprises information describing one or more snapshots of the dataset and a mapping of content segments to one or more addresses associated with the dataset; and
updating, using the metadata update, metadata of the dataset on another storage system.
2 . The method of claim 1 , wherein the cloud-based storage system includes a plurality of cloud computing instances that respectively store a portion of the dataset.
3 . The method of claim 1 , wherein the cloud-based storage system includes a cloud-based object storage that stores all portions of the dataset.
4 . The method of claim 1 , wherein a metadata representation of the dataset includes one or more references to one or more storage locations within one or more of the storage systems.
5 . The method of claim 1 , wherein a metadata representation of the dataset on a first storage system is structured differently than the metadata representation of the dataset on a second storage system.
6 . The method of claim 1 , further comprising:
responsive to successfully applying the I/O operation on the cloud-based storage system, updating, on the cloud-based storage system and in accordance with the metadata update, a version of metadata corresponding to the dataset prior to applying the I/O operation.
7 . The method of claim 1 , wherein the metadata update includes a description of an update to be made to represent application of the I/O operation.
8 . The method of claim 1 , wherein the dataset is included in one or more volumes.
9 . The method of claim 1 , wherein the one or more snapshots are created using metadata objects.
10 . The method of claim 1 , wherein the metadata update further includes deduplication information describing a manner in which an I/O operation payload is to be included in relation to previously stored data within the dataset.
11 . A cloud-based storage system in a cloud computing environment, the cloud-based storage system including:
one or more cloud computing instances with local block storage;
cloud-based object storage; and
one or more storage controller applications, each storage controller application executing in a cloud computing instance, wherein the one or more storage controllers are configured to:
generating for a dataset on a cloud-based storage system, based on an I/O operation directed to the dataset, a metadata update that comprises information describing one or more snapshots of the dataset and a mapping of content segments to one or more addresses associated with the dataset; and
updating, using the metadata update, metadata of the dataset on another storage system.
12 . The cloud-based storage system of claim 11 , wherein the cloud-based storage system includes a plurality of cloud computing instances that respectively store a portion of the dataset.
13 . The cloud-based storage system of claim 11 , wherein the cloud-based storage system includes a cloud-based object storage that stores all portions of the dataset.
14 . The cloud-based storage system of claim 11 , wherein a metadata representation of the dataset includes one or more references to one or more storage locations within one or more of the storage systems.
15 . The cloud-based storage system of claim 11 , wherein a metadata representation of the dataset on a first storage system is structured differently than the metadata representation of the dataset on a second storage system.
16 . The cloud-based storage system of claim 11 , wherein the one or more storage controller applications are further configured to:
responsive to successfully applying the I/O operation on the cloud-based storage system, updating, on the cloud-based storage system and in accordance with the metadata update, a version of metadata corresponding to the dataset prior to applying the I/O operation.
17 . The cloud-based storage system of claim 11 , wherein the metadata update includes a description of an update to be made to represent application of the I/O operation.
18 . The cloud-based storage system of claim 11 , wherein the one or more snapshots are created using metadata objects.
19 . The cloud-based storage system of claim 11 , wherein the metadata update further includes deduplication information describing a manner in which an I/O operation payload is to be included in relation to previously stored data within the dataset.
20 . A computer program product disposed on a non-transitory computer readable medium, the computer program product including computer program instructions that, when executed, carry out the steps of:
generating for a dataset on a cloud-based storage system, based on an I/O operation directed to the dataset, a metadata update that comprises information describing one or more snapshots of the dataset and a mapping of content segments to one or more addresses associated with the dataset; and
updating, using the metadata update, metadata of the dataset on another storage system.