Object store mirroring and resync, during garbage collection operation, first bucket (with deleted first object) with second bucket
Techniques are provided for object store mirroring. Data within a storage tier of a node may be determined as being data to tier out to a primary object store based upon a property of the data. A first object is generated to comprise the data. A second object is generated to comprise the data. The first object is transmitted to the primary data store for storage in parallel with the second object being transmitted to a mirror object store for storage. Tiering of the data is designated as successful once acknowledgements are received from both the primary object that the first object was stored and the mirror object store that the second object was stored.
1. A method comprising:
generating a first object and a second object to comprise data of a storage tier of a node based upon a property of the data;
transmitting, in parallel, the first object to a primary object store for storage within a first storage bucket and the second object to a mirror object store for storage within a second storage bucket;
performing a garbage collection process to transmit a first delete operation to the primary object store in parallel with a second delete operation to the mirror object store, wherein the first delete operation targets the first object within the primary object store;
and during the garbage collection process, performing a resync operation of the first storage bucket with the second storage bucket, wherein the first object is deleted within the first storage bucket.
2. The method of claim 1 , wherein the garbage collection process is performed based upon a determination that the data of the storage tier, stored within the first object and the second object, is no longer referenced by a file system of the node.
3. The method of claim 1 , wherein the second delete operation targets the second object within the mirror object store.
4. The method of claim 1 , wherein the first storage bucket comprises a set of objects storing the data of the storage tier and the second storage bucket is initially created as a new empty storage bucket.
5. The method of claim 1 , comprising:
determining that the garbage collection process is successful based upon the first delete operation and the second delete operation succeeding.
6. The method of claim 1 , comprising:
determining that the garbage collection process is unsuccessful based upon the first delete operation succeeding and the second delete operation failing.
7. The method of claim 1 , comprising:
determining that the garbage collection process is unsuccessful based upon the first delete operation failing and the second delete operation succeeding.
8. The method of claim 1 , comprising:
skipping, by the resync operation, an object within the first storage bucket.
9. The method of claim 8 , comprising:
skipping, by the resync operation, an object within the first storage bucket having a zero reference count.
10. The method of claim 8 , comprising:
skipping, by the resync operation, an object within the first storage bucket having a creating state.
11. The method of claim 8 , wherein the primary object store is hosted in a cloud computing environment.
12. The method of claim 1 , comprising:
skipping, by the resync operation, objects within the first storage bucket having a zero reference count and objects within the first storage bucket having a creating state.
13. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:
generate a first object and a second object to comprise data of a storage tier of a node based upon a property of the data;
transmit, in parallel, the first object to a primary object store for storage within a first storage bucket and the second object to a mirror object store for storage within a second storage bucket;
perform a garbage collection process to transmit a first delete operation to the primary object store in parallel with a second delete operation to the mirror object store, wherein the first delete operation targets the first object within the primary object store;
and during the garbage collection process, performing a resync operation of the first storage bucket with the second storage bucket, wherein the first object is deleted within the first storage bucket.
14. The non-transitory machine readable medium of claim 13 , comprising:
skipping, by the resync operation, an object within the first storage bucket.
15. The non-transitory machine readable medium of claim 13 , wherein the second delete operation targets the second object within the mirror object store.
16. The non-transitory machine readable medium of claim 13 , wherein the garbage collection process is performed based upon a determination that the data of the storage tier, stored within the first object and the second object, is no longer referenced by a file system of the node.
17. A computing device comprising:
a memory comprising machine executable code;
and a processor coupled to the memory, the processor configured to execute the machine executable code to cause the computing device to:
generate a first object and a second object to comprise first data of a storage tier of a node based upon a property of the first data;
transmit, in parallel, the first object to a primary object store for storage within a first storage bucket and a second object to a mirror object store for storage within a second storage bucket;
perform a garbage collection process to transmit a first delete operation to the primary object store in parallel with a second delete operation to the mirror object store, wherein the first delete operation targets the first object within the primary object store;
and during the garbage collection process, performing a resync operation of the first storage bucket with the second storage bucket, wherein the first object is deleted within the first storage bucket.
18. The computing device of claim 17 , wherein the second delete operation targets the second object within the mirror object store.
19. The computing device of claim 17 , wherein the machine executable code causes the computing device to:
perform the garbage collection process based upon a determination that the data of the storage tier stored within the first object and the second object is no longer referenced by a file system of the node.
20. The computing device of claim 17 further comprising skipping, by the resync operation, an object within the first storage bucket having a zero reference count.