Extending retention lock protection from on-premises to the cloud
Embodiments for retention locking a deduplicated file stored in cloud storage by defining object metadata for each object of the file, and comprising a lock count and a retention time based on an expiry date of the lock, with each object having segments, the object metadata further having a respective expiry date and lock count for each segment, where at least some segments are shared among two or more files. Also updating the lock count and retention time for all segments of the file being locked; and if the object is not already locked, locking the object using a retention lock defining a retention time and updating the object metadata with a new lock count and the retention time, otherwise incrementing the lock count and updating the retention time for the expiry date if expiry date of a previous lock is older than a current expiry date.
1. A computer-implemented method to retention lock a file stored in cloud storage by a deduplication backup system, comprising:
defining object metadata for each object of the file, and comprising a lock count and a retention time based on an expiry date of the retention lock, wherein each object comprises a plurality of segments, the object metadata further comprising a respective expiry date and lock count for each segment of the object, wherein the expiry date metadata comprises a maximum expiry date among all the segments;
identifying all objects that contain segments of the file being locked;
updating, for each object, the lock count and retention time in the metadata object for all segments of the file being locked, wherein the lock count metadata comprises a sum of the lock counts of all the segments;
determining if an object is already locked,
if the object is not already locked, locking the object using a retention lock defining a certain retention time and updating the object metadata with a new lock count and the retention time;
if the object is already locked, incrementing the lock count and updating the retention time for the expiry date if expiry date of a previous lock is older than a current expiry date;
selecting, through a retention lock mode, one of retention lock governance (RLG) and retention lock compliance (RLC); and
reverting the RLG by decrementing the lock count of the object and on each segment of the object, reassessing the segments of the object to find a remaining maximum expiration date, and locking the object again for a duration of the maximum expiry date, if the remaining maximum expiry date is not less than the remaining maximum expiration date.
2. The method of claim 1 wherein the object is locked using a retention lock software process provided by a cloud provider supporting the cloud storage through an application programming interface (API).
3. The method of claim 1 wherein the deduplication backup system produces deduplicated segments within the object with different respective retention times.
4. The method of claim 1 wherein the locking step is performed by one of a synchronous process and an asynchronous process with a callback function.
5. The method of claim 1 wherein the RLG retention lock can be reverted, and the RLC retention lock cannot be reverted.
6. The method of claim 5 wherein the locking is performed by temporarily unlocking the object and re-locking it with the remaining maximum expiry date.
7. The method of claim 6 further comprising:
deleting the expiry date of a segment when the lock count of the segment is decremented to zero; and
reverting the RLG lock when the lock count of the object is decremented to zero.
8. The method of claim 5 wherein the object metadata for the object further comprises a lock mode indicator, and wherein the step of determining if the object is already locked further comprises determining if the object is already locked in a different lock mode between the RLG and RLC lock modes, and if so, copying the object to a new object locked by the new lock mode.
9. The method of claim 8 the method further comprising: updating the lock mode indicator of the copied object with the new lock mode, and updating the retention time of the copied object with a new retention time for the new lock mode.
10. The method of claim 9 wherein the object is copied to the new object when threshold values are met for a lock time of an original RLC retention lock and an amount of data shared between the RLG and RLC lock modes.
11. The method of claim 9 wherein the object is copied to the new object when certain heuristic measurements between the RLG and RLC lock modes are met.
12. The method of claim 1 wherein the cloud storage comprises storage media resident in a cloud computing network maintained by a cloud service provider, and provided for long term retention of data processed using the metadata, and further wherein the deduplication backup system is executed by a data storage server running a Data Domain file system (DDFS).
13. A tangible computer program product having stored thereon program code that when executed by a processor, cause the processor to perform a method of retention locking a file stored in cloud storage by a deduplication backup system comprising:
defining object metadata for each object of the file, and comprising a lock count and a retention time based on an expiry date of the retention lock, wherein each object comprises a plurality of segments, the object metadata further comprising a respective expiry date and lock count for each segment of the object, wherein the expiry date metadata comprises a maximum expiry date among all the segments;
identifying all objects that contain segments of the file being locked;
updating, for each object, the lock count and retention time in the metadata object for all segments of the file being locked, wherein the lock count metadata comprises a sum of the lock counts of all the segments;
determining if an object is already locked, if the object is not already locked, locking the object using a retention lock defining a certain retention time and updating the object metadata with a new lock count and the retention time;
if the object is already locked, incrementing the lock count and updating the retention time for the expiry date if expiry date of a previous lock is older than a current expiry date;
selecting, through a retention lock mode, one of retention lock governance (RLG) and retention lock compliance (RLC); and
reverting the RLG by decrementing the lock count of the object and on each segment of the object, reassessing the segments of the object to find a remaining maximum expiration date, and locking the object again for a duration of the maximum expiry date, if the remaining maximum expiry date is not less than the remaining maximum expiration date.