Garbage collection for a deduplicated cloud tier with encrypted segments

1. In a computing system that provides an active tier of data and a cloud tier of data, a method for performing a garbage collection operation in the cloud tier, the method comprising:

processing metadata stored in the active tier for the cloud tier by a garbage collection engine at an on-premise system associated with the active tier to identify containers in the cloud tier to be cleaned, wherein the identified containers contain compression regions, wherein at least some of the compression regions include both dead segments and live segments that are encrypted;

generating recipes from the metadata stored at the active tier that identify locations of the live segments and of the dead segments in the compression regions in the identified containers stored in the cloud tier;

initiating, by the garbage collection engine, a microservice in a cloud;

sending the recipes from the garbage collection engine operating at the on-premise system to the microservice operating in a cloud;

performing the recipes by the microservice, wherein the microservice exits after the garbage collection operation ends, wherein the microservice:

decrypts the compression regions in the identified containers including the compression regions that include both dead segments and live segments, and copies only the live segments to compression regions in new containers from the identified containers;

encrypts the live segments in the compression regions in the new containers; and

deletes the identified containers after the recipes have been performed to free storage space used by the identified containers;

copying metadata corresponding to the recipes into a new metadata container at the active tier and replicating the new metadata container to the cloud tier.

2. The method of claim 1 , wherein the recipes identify at least a container, a location of the live segments or of the dead segments in the compression regions included in the container, and a destination container for storing the live segments.

3. The method of claim 1 , further comprising decompressing the compression regions prior to copying forward the live segments and compressing the live segments in the compression regions of the new containers.

4. The method of claim 1 , further comprising updating the metadata stored in the active tier to reflect the cloud tier after performing the recipes.

5. The method of claim 1 , further comprising transmitting at least one key to the microservice, wherein the at least one key is used to decrypt the compression regions prior to copying forward the live segments and to encrypt the live segments in the new compression regions after copying forward the live segments.

6. The method of claim 5 , further comprising identifying metadata of L0 and Lp containers included in the metadata for the cloud tier, the metadata of the L0 and Lp containers including fingerprints of segments in the L0 and Lp containers.

7. The method of claim 6 , further comprising performing a lookup to identify live segments and dead segments of the Lp containers.

8. The method of claim 7 , further comprising generating the recipes that allow the live segments from the Lp containers to be copied into new LP containers.

9. The method of claim 8 , further comprising writing the new Lp containers locally and to the cloud.

10. The method of claim 9 , further comprising copying metadata of the new Lp containers to a new CMETA container, wherein the new CMETA container is written locally and to the cloud.

11. The method of claim 1 , further comprising iterating metadata sections of local CMETA containers to identify live segments of L0 containers.

12. The method of claim 11 , further comprising forming the recipes based on the local CMETA containers.

13. The method of claim 12 , further comprising copying metadata corresponding to the recipes into a new CMETA container locally and replicating the new CMETA container to the cloud.

14. A non-transitory computer readable medium comprising computer executable instructions that, when executed perform a garbage collection operation in a cloud tier of data associated with a computing system that also has a local tier of data, the method comprising:

processing metadata stored in the active tier for the cloud tier by a garbage collection engine at an on-premise system associated with the active tier to identify containers in the cloud tier to be cleaned, wherein the identified containers contain compression regions, wherein at least some of the compression regions include both dead segments and live segments that are encrypted;

generating recipes from the metadata stored at the active tier that identify locations of the live segments and of the dead segments in the compression regions in the identified containers stored in the cloud tier;

initiating a microservice in a cloud;

sending the recipes from the garbage collection engine operating at the on-premise system to the microservice operating in a cloud;

performing the recipes by the microservice, wherein the microservice is spun down after the garbage collection operation ends, wherein the microservice:

decrypts the compression regions in the identified containers including the compression regions that include both dead segments and live segments, and copies only the live segments to compression regions in new containers from the identified containers;

encrypts the live segments in the compression regions in the new containers; and

deletes the identified containers after the recipes have been performed to free storage space used by the identified containers; and

copying metadata corresponding to the recipes into a new metadata container at the active tier and replicating the new metadata container to the cloud tier.

15. The non-transitory computer readable medium of claim 14 , wherein the recipes identify at least a container, a location of the live segments and the dead segments in the compression regions included in the container, and a destination container for storing the live segments.

16. The non-transitory computer readable medium of claim 14 , further comprising decompressing the compression regions prior to copying forward the live segments and compressing the live segments in the compression regions of the new containers.

17. The non-transitory computer readable medium of claim 14 , further comprising transmitting at least one key to the microservice, wherein the at least one key is used to decrypt the compression regions prior to copying forward the live segments and to encrypt the live segments in the new compression regions after copying forward the live segments.

18. The non-transitory computer readable medium of claim 17 , further comprising:

identifying metadata of L0 and Lp containers included in the metadata for the cloud tier, the metadata of the L0 and Lp containers including fingerprints of segments in the L0 and Lp containers;

performing a lookup to identify live segments and dead segments of the Lp containers;

generating the recipes that allow only

the live segments from the Lp containers to be copied into new LP containers;

writing the new Lp containers locally and to the cloud; and

copying metadata of the new Lp containers to a new CMETA container, wherein the new CMETA container is written locally and to the cloud.

19. The method of claim 1 , further comprising:

iterating metadata sections of local CMETA containers to identify live segments of L0 containers;

forming the recipes based on the local CMETA containers; and

copying metadata corresponding to the recipes into a new CMETA container locally and replicating the new CMETA container to the cloud.