Synchronously replicating datasets and other managed objects to cloud-based storage systems
A pod, the pod including the dataset, a set of managed objects and management operations, a set of access operations to modify or read the dataset, and a plurality of storage systems, where: management operations can modify or query managed objects equivalently through any of the storage systems, access operations to read or modify the dataset operate equivalently through any of the storage systems, each storage system stores a separate copy of the dataset as a proper subset of the datasets stored and advertised for use by the storage system, and operations to modify managed objects or the dataset performed and completed through any one storage system are reflected in subsequent management objects to query the pod or subsequent access operations to read the dataset.
1. A plurality of storage systems across which a dataset is synchronously replicated, each storage system including a computer memory and a computer processor, the computer memory in each of the storage systems including computer program instructions that, when executed by the computer processor of a particular storage system, cause the particular storage system to carry out the steps of:
attaching to a pod, wherein the pod includes the dataset, a set of managed objects and management operations, a set of access operations to modify or read the dataset, and a plurality of member storage systems, and wherein:
management operations modify or query managed objects equivalently through any of the member storage systems,
access operations to read or modify the dataset operate equivalently through any of the member storage systems,
each member storage system stores a separate copy of the dataset as a proper subset of a plurality of datasets stored and advertised for use by the storage system, and
operations that modify managed objects or the dataset, performed and completed through any one member storage system, are reflected in subsequent management objects to query the pod or subsequent access operations to read the dataset.
2. The plurality of storage systems of claim 1 , wherein one or more of the storage systems include computer program instructions that, when executed by the computer processor of a particular storage system, cause the particular storage system to carry out the step of:
receiving a request to read a portion of the dataset; and
processing the request to read the portion of the dataset locally.
3. The plurality of storage systems of claim 1 , wherein one or more of the storage systems include computer program instructions that, when executed by the computer processor of a particular storage system, cause the particular storage system to carry out the step of:
detecting a disruption in data communications with one or more of the other storage systems;
determining whether the particular storage system should remain in the pod;
responsive to determining that the particular storage system should remain in the pod, keeping the dataset on the particular storage system accessible for management and dataset operations; and
responsive to determining that the particular storage system should not remain in the pod, making the dataset on the particular storage system inaccessible for management and dataset operations.
4. The plurality of storage systems of claim 3 , wherein one or more of the storage systems include computer program instructions that, when executed by the computer processor of a particular storage system, cause the particular storage system to carry out the step of:
detecting that the disruption in data communications with one or more of the other storage systems has been repaired; and
making the dataset on the particular storage system accessible for management and dataset operations.
5. The plurality of storage systems of claim 1 , wherein two or more of the storage systems include computer program instructions that, when executed by the computer processor of each storage system, cause each storage system to carry out the step of:
identifying a target storage system for asynchronously receiving the dataset, wherein the target storage system is not one of the plurality of storage systems across which the dataset is synchronously replicated;
identifying a portion of the dataset that is not being asynchronously replicated to the target storage system by any of the other storages systems; and
asynchronously replicating, to the target storage system, the portion of the dataset that is not being asynchronously replicated to the target storage system by any of the other storages systems, wherein the two or more storage systems collectively replicate the entire dataset to the target storage system.
6. The plurality of storage systems of claim 1 , wherein at least one of the storage systems is embodied as cloud storage that is provided by a cloud services provider.
7. A method of synchronously replicating a dataset across a plurality of storage systems, the method comprising:
attaching, by the plurality of storage systems, to a pod, wherein the pod includes the dataset, a set of managed objects and management operations, a set of access operations to modify or read the dataset, and a plurality of member storage systems, and wherein:
management operations modify or query managed objects equivalently through any of the member storage systems,
access operations to read or modify the dataset operate equivalently through any of the member storage systems,
each member storage system stores a separate copy of the dataset as a proper subset of a plurality of datasets stored and advertised for use by the storage system, and
operations that modify managed objects or the dataset, performed and completed through any one member storage system, are reflected in subsequent management objects to query the pod or subsequent access operations to read the dataset.
8. The method of claim 7 further comprising:
receiving, by a particular storage system that is one of the plurality of storage systems, a request to read a portion of the dataset; and
processing, by the particular storage system, the request to read the portion of the dataset locally.
9. The method of claim 7 further comprising:
detecting, by a particular storage system that is one of the plurality of storage systems, a disruption in data communications with one or more of the other storage systems in the plurality of storage systems;
determining whether the particular storage system should remain in the pod;
responsive to determining that the particular storage system should remain in the pod, keeping the dataset on the particular storage system accessible for management and dataset operations; and
responsive to determining that the particular storage system should not remain in the pod, making the dataset on the particular storage system inaccessible for management and dataset operations.
10. The method of claim 9 further comprising:
detecting that the disruption in data communications with one or more of the other storage systems has been repaired; and
making the dataset on the particular storage system accessible for management and dataset operations.
11. The method of claim 7 further comprising:
identifying a target storage system for asynchronously receiving the dataset, wherein the target storage system is not one of the plurality of storage systems across which the dataset is synchronously replicated;
identifying a portion of the dataset that is not being asynchronously replicated to the target storage system by any of the other storages systems; and
asynchronously replicating, to the target storage system, the portion of the dataset that is not being asynchronously replicated to the target storage system by any of the other storages systems, wherein the two or more storage systems collectively replicate the entire dataset to the target storage system.
12. The method of claim 7 wherein at least one of the storage systems is embodied as cloud storage that is provided by a cloud services provider.
13. An apparatus for synchronously replicating a dataset across a plurality of storage systems, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
attaching to a pod, wherein the pod includes the dataset, a set of managed objects and management operations, a set of access operations to modify or read the dataset, and a plurality of member storage systems, and wherein:
management operations modify or query managed objects equivalently through any of the member storage systems,
access operations to read or modify the dataset operate equivalently through any of the member storage systems,
each member storage system stores a separate copy of the dataset as a proper subset of a plurality of datasets stored and advertised for use by the storage system, and
operations that modify managed objects or the dataset performed and completed through any one member storage system, are reflected in subsequent management objects to query the pod or subsequent access operations to read the dataset.
14. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
receiving a request to read a portion of the dataset; and
processing the request to read the portion of the dataset locally.
15. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
detecting a disruption in data communications with one or more of the other storage systems;
determining whether the particular storage system should remain in the pod;
responsive to determining that the particular storage system should remain in the pod, keeping the dataset on the particular storage system accessible for management and dataset operations; and
responsive to determining that the particular storage system should not remain in the pod, making the dataset on the particular storage system inaccessible for management and dataset operations.
16. The apparatus of claim 15 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
detecting that the disruption in data communications with one or more of the other storage systems has been repaired; and
making the dataset on the particular storage system accessible for management and dataset operations.
17. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
identifying a target storage system for asynchronously receiving the dataset, wherein the target storage system is not one of the plurality of storage systems across which the dataset is synchronously replicated;
identifying a portion of the dataset that is not being asynchronously replicated to the target storage system by any of the other storages systems; and
asynchronously replicating, to the target storage system, the portion of the dataset that is not being asynchronously replicated to the target storage system by any of the other storages systems, wherein the two or more storage systems collectively replicate the entire dataset to the target storage system.