Using data mirroring across multiple regions to reduce the likelihood of losing objects maintained in cloud object storage
Techniques for using data mirroring across regions to reduce the likelihood of losing objects in a cloud object storage platform are provided. In one set of embodiments, a computer system can upload first and second copies of a data object to first and second regions of the cloud object storage platform respectively, where the first and second copies are identical. The computer system can then attempt to read the first copy of the data object from the first region. If the read attempt fails, the computer system can retrieve the second copy of the data object from the second region.
1. A method comprising:
identifying, by a first garbage collector component running at a first region of a cloud object storage platform, a first data object stored at the first region as a candidate for garbage collection, wherein the first data object is a mirror copy of a second data object stored at a second region of the cloud object storage platform;
determining, by the first garbage collector component, a range of live data blocks in the first data object, the determining being based on a metadata database residing at the first region that holds metadata for the first and second data objects;
reading, by the first garbage collector component, content of the range of live data blocks from the first data object;
writing, by the first garbage collector component, the content of the range of live data blocks as a third data object in the first region; and
transmitting, by the first garbage collector component to a second garbage collector component running at the second region, garbage collection metadata including an identifier of the first data object and the range of live data blocks.
2. The method of claim 1 wherein the garbage collection metadata excludes the content of the range of live data blocks.
3. The method of claim 1 wherein the first and second data objects share a common name, and wherein the identifier of the first data object included in the garbage collection metadata is the common name.
4. The method of claim 1 wherein the identifying comprises:
determining that a percentage of live data blocks in the first data object is below a predefined threshold.
5. The method of claim 1 wherein upon receiving the garbage collection metadata, the second garbage collector component:
reads content of the range of live data blocks from the second data object;
writes the content of the range of live data blocks read from the second data object as a fourth data object in the second region; and
marks the second data object as deleted at the second region.
6. The method of claim 1 further comprising:
marking the first data object as deleted at the first region.
7. The method of claim 1 wherein the first and second regions correspond to first and second sets of servers of the cloud object storage platform respectively that are located in different geographic areas.
8. A non-transitory computer readable storage medium having stored thereon program code executable by a first server running at a first region of a cloud object storage platform, the program code embodying a method comprising:
identifying a first data object stored at the first region as a candidate for garbage collection, wherein the first data object is a mirror copy of a second data object stored at a second region of the cloud object storage platform;
determining a range of live data blocks in the first data object, the determining being based on a metadata database residing at the first region that holds metadata for the first and second data objects;
reading content of the range of live data blocks from the first data object;
writing the content of the range of live data blocks as a third data object in the first region; and
transmitting, to a second server running at the second region, garbage collection metadata including an identifier of the first data object and the range of live data blocks.
9. The non-transitory computer readable storage medium of claim 8 wherein the garbage collection metadata excludes the content of the range of live data blocks.
10. The non-transitory computer readable storage medium of claim 8 wherein the first and second data objects share a common name, and wherein the identifier of the first data object included in the garbage collection metadata is the common name.
11. The non-transitory computer readable storage medium of claim 8 wherein the identifying comprises:
determining that a percentage of live data blocks in the first data object is below a predefined threshold.
12. The non-transitory computer readable storage medium of claim 8 wherein upon receiving the garbage collection metadata, the second server:
reads content of the range of live data blocks from the second data object;
writes the content of the range of live data blocks read from the second data object as a fourth data object in the second region; and
marks the second data object as deleted at the second region.
13. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:
marking the first data object as deleted at the first region.
14. The non-transitory computer readable storage medium of claim 8 wherein the first and second regions correspond to first and second sets of servers of the cloud object storage platform respectively that are located in different geographic areas.
15. A computer system running within a first region of a cloud object storage platform, the computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that, when executed, causes the processor to:
identify a first data object stored at the first region as a candidate for garbage collection, wherein the first data object is a mirror copy of a second data object stored at a second region of the cloud object storage platform;
determine a range of live data blocks in the first data object, the determining being based on a metadata database residing at the first region that holds metadata for the first and second data objects;
read content of the range of live data blocks from the first data object;
write the content of the range of live data blocks as a third data object in the first region; and
transmit, to another computer system running at the second region, garbage collection metadata including an identifier of the first data object and the range of live data blocks.
16. The computer system of claim 15 wherein the garbage collection metadata excludes the content of the range of live data blocks.
17. The computer system of claim 15 wherein the first and second data objects share a common name, and wherein the identifier of the first data object included in the garbage collection metadata is the common name.
18. The computer system of claim 15 wherein the program code that causes the processor to identify the first data object as a candidate for garage collection comprises program code that causes the processor to:
determine that a percentage of live data blocks in the first data object is below a predefined threshold.
19. The computer system of claim 15 wherein upon receiving the garbage collection metadata, said another computer system:
reads content of the range of live data blocks from the second data object;
writes the content of the range of live data blocks read from the second data object as a fourth data object in the second region; and
marks the second data object as deleted at the second region.
20. The computer system of claim 15 wherein the program code further causes the processor to:
mark the first data object as deleted at the first region.
21. The computer system of claim 15 wherein the first and second regions correspond to first and second sets of computer systems of the cloud object storage platform respectively that are located in different geographic areas.