Synthetically providing multiple types of granular delete operations for a cloud-based object storage
Described is a system (and method) for providing multiple types of granular delete operations for a cloud-based object storage. The system may include a server that acts as an intermediary between a client device and an object storage that stores client data. The server may allow a client to perform a hard delete (or permanent) delete and a soft delete. The server may perform a specialized processing to synthetically create the soft delete feature using a permanent object delete operation provided by the object storage. The specialized processing may include manipulating retention periods associated with objects depending on whether the object is subject to a hard delete or a soft delete. As a result, the server may provide the ability for clients to perform a soft delete at an object level using existing storage level APIs that do not directly support such a feature.
1. A system comprising:
one or more processors; and
a non-transitory computer-readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
receive, via a server application programming interface (API) at a server, a request to delete an object stored on an object storage using a one of a plurality of API methods from a client device, the request indicating a type of delete as being either a hard delete or a soft delete;
determine, by the server, whether the type of delete in the request is a hard delete or a soft delete based on the API method used;
update, by the server and based on the indicated delete, a retention period field value for a retention period associated with the object and stored within an object catalog based on the determination of the type of delete in the request, the retention period field value updated to a present point in time so the object will be identified and deleted if the request indicates the hard delete, and the retention period field value updated to a future point in time if the request indicates the soft delete, the object catalog being a data structure comprising fields associated with objects stored on the object storage, a retention period field being one of a plurality of fields within the object catalog, where each object referenced in the object catalog has a value associated with the retention period field;
monitor, periodically by the server, the object catalog, to identify objects associated with an expired value in the retention period field, the identified objects including at least the object identified during at least one periodic monitoring; and
initiate, periodically by the server, a standard delete operation provided by a storage API at the object storage to permanently delete the identified objects from the object storage after a corresponding expired retention period.
2. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
update, by the server and if the request to delete the object indicates the soft delete, the object catalog to indicate the object is marked for deletion.
3. The system of claim 2 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
receive, by the server and from the client device, a request to obtain a list of objects stored on the object storage; and
provide, by the server, the list of objects stored on the object storage, the list not including the object based on the object catalog indicating the object is marked for deletion.
4. The system of claim 1 , wherein if the request indicates the soft delete, the request includes the future point in time as a value for the retention period field within the object catalog for the object.
5. The system of claim 4 , wherein the soft delete is indicated based on the request being received via a soft delete method of the server API provided by the server, the soft delete method configured to receive the value for the retention period field within the object catalog.
6. The system of claim 1 , wherein initiating the standard delete operation includes accessing a delete method of the storage API provided by the object storage, the delete method configured to permanently delete the identified objects irrespective of whether a particular identified object is associated with the hard delete or the soft delete.
7. The system of claim 1 , wherein the delete operation provided by the object storage is an only object-level delete operation provided by the object storage.
8. The system of claim 1 , wherein the at least one periodic monitoring includes monitoring the object catalog immediately after receiving the request to delete the object.
9. A method comprising:
receiving, via a server application programming interface (API) at a server, a request to delete an object stored on an object storage using a one of a plurality of API methods from a client device, the request indicating a type of delete as being either a hard delete or a soft delete;
determining, at the server, whether the type of delete in the request is a hard delete or a soft delete based on the API method used;
updating, by the server and based on the indicated delete, a retention period field value for a retention period associated with the object and stored within an object catalog based on the determination of the type of delete in the request, the retention period field value updated to a present point in time so the object will be identified and deleted if the request indicates the hard delete, and the retention period field value updated to a future point in time if the request indicates the soft delete, the object catalog being a data structure comprising fields associated with objects stored on the object storage, a retention period field being one of a plurality of fields within the object catalog, where each object referenced in the object catalog has a value associated with the retention period field;
monitoring, periodically by the server, the object catalog, identify objects associated with an expired value in the retention period field, the identified objects including at least the object identified during at least one periodic monitoring; and
initiating, periodically by the server, a standard delete operation provided by a storage API at the object storage to permanently delete the identified objects from the object storage after a corresponding expired retention period.
10. The method of claim 9 , further comprising:
updating, by the server and if the request to delete the object indicates the soft delete, the object catalog to indicate the object is marked for deletion.
11. The method of claim 10 , further comprising:
receiving, by the server and from the client device, a request to obtain a list of objects stored on the object storage; and
providing, by the server, the list of objects stored on the object storage, the list not including the object based on the object catalog indicating the object is marked for deletion.
12. The method of claim 9 , wherein if the request indicates the soft delete, the request includes the future point in time as a value for the retention period field within the object catalog for the object.
13. The method of claim 12 , wherein the soft delete is indicated based on the request being received via a soft delete method of the server API provided by the server, the soft delete method configured to receive the value for the retention period field within the object catalog.
14. The method of claim 9 , wherein initiating the delete operation includes accessing a delete method of the storage API provided by the object storage, the delete method configured to permanently delete the identified objects irrespective of whether a particular identified object is associated with the hard delete or the soft delete.
15. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
receive, via a server application programming interface (API) at a server, a request to delete an object stored on an object storage using a one of a plurality of API methods from a client device, the request indicating a type of the delete as being either a hard delete or a soft delete;
determine, by the server, whether the type of delete in the request is a hard delete or a soft delete based on the API method used;
update, by the server and based on the indicated delete, a retention period field value for a retention period associated with the object and stored within an object catalog based on the determination of the type of delete in the request, the retention period field value updated to a present point in time so the object will be identified and deleted if the request indicates the hard delete, and the retention period field value updated to a future point in time if the request indicates the soft delete, the object catalog being a data structure comprising fields associated with objects stored on the object storage, a retention period field being one of a plurality of fields within the object catalog, where each object referenced in the object catalog has a value associated with the retention period field;
monitor, periodically by the server, the object catalog, to identify objects associated with an expired value in the retention period field, the identified objects including at least the object identified during at least one periodic monitoring; and
initiate, periodically by the server, a standard delete operation provided by a storage API at the object storage to permanently delete the identified objects from the object storage after a corresponding expired retention period.
16. The computer program product of claim 15 , wherein the program code includes further instructions to:
update, by the server and if the request to delete the object indicates the soft delete, the object catalog to indicate the object is marked for deletion.
17. The computer program product of claim 16 , wherein the program code includes further instructions to:
receive, by the server and from the client device, a request to obtain a list of objects stored on the object storage; and
provide, by the server, the list of objects stored on the object storage, the list not including the object based on the object catalog indicating the object is marked for deletion.
18. The computer program product of claim 15 , wherein if the request indicates the soft delete, the request includes the future point in time as a value for the retention period field within the object catalog for the object.
19. The computer program product of claim 18 , wherein the soft delete is indicated based on the request being received via a soft delete method of the server API provided by the server, the soft delete method configured to receive the value for the retention period field within the object catalog.
20. The computer program product of claim 15 , wherein initiating the delete operation includes accessing a delete method of the storage API provided by the object storage, the delete method configured to permanently delete the identified objects irrespective of whether a particular identified object is associated with the hard delete or the soft delete.