Efficient access-based reallocation of backup data within an object storage
Described is a system (and method) for efficient access-based reallocation of backup data stored within an object storage. The system may implement various specialized procedures to efficiently store and manage backup data within the object storage. These procedures may include packing backup data into objects to improve data operation efficiency. The system may also conserve storage space on the object storage by performing an analysis to reallocate backup data as necessary. For example, data may be stored in objects in an immutable manner, and thus, the system may efficiently reallocate data to new objects based on data access patterns. For example, the system may determine an access pattern associated with live data remaining within the first object satisfies a condition, and in response, reallocated the live data to a new object.
1. A system comprising:
one or more processors; and
a non-transitory computer-readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
perform a first backup of client data stored on a client device at a first point-in-time to an object storage, the client data at the first point-in-time including a first file and a second file, the first backup including creating a first object within the object storage to store a backup of the first file and the second file;
perform, after the first backup, a second backup of the client data stored on the client device at a second point-in-time to the object storage, the client data at the second point-in-time including a new third file and a deletion of the first file, the second backup including creating a second object within the object storage to store a backup of the new third file;
update a metadata database associated with the object storage to indicate the backup of the first file stored within the first object represents deleted data, in response to the deletion of the first file;
determine whether an access pattern associated with live data remaining within the first object satisfies one or more conditions, the live data remaining within the first object including the backup of the second file; and
reallocate the live data remaining within the first object by creating a third object within the object storage to store the backup of the second file, in response to determining the access pattern associated with the live data satisfies the condition.
2. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
determine whether an amount of the live data remaining within the first object falls below a predetermined threshold, wherein
reallocating the live data is further in response to determining the amount of the live data remaining within the first object falls below the predetermined threshold.
3. The system of claim 2 , wherein the predetermined threshold is based on an amount of the live data remaining within the first object compared to an amount of the deleted data within the first object.
4. The system of claim 1 , wherein the one or more conditions include:
the backup of the second file stored on the object storage being accessed a predetermined number of times within a given time period.
5. The system of claim 1 , wherein the one or more conditions include:
the second file stored on the client device being accessed a predetermined number of times within a given time period.
6. The system of claim 1 , wherein the one or more conditions include:
a predetermined amount of time elapsing since storing the backup of the second file on the object storage.
7. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
update the metadata database to indicate the backup of the second file is stored within the third object, in response to reallocating the live data.
8. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
update the metadata database to indicate the backup of the first file and the backup of the second file are stored within the first object, in response to performing the first backup; and
update the metadata database to indicate the backup of the third file is stored within the second object, in response to performing the second backup.
9. A method comprising:
performing a first backup of client data stored on a client device at a first point-in-time to an object storage, the client data at the first point-in-time including a first file and a second file, the first backup including creating a first object within the object storage to store a backup of the first file and the second file;
performing, after the first backup, a second backup of the client data stored on the client device at a second point-in-time to the object storage, the client data at the second point-in-time including a new third file and a deletion of the first file, the second backup including creating a second object within the object storage to store a backup of the new third file;
updating a metadata database associated with the object storage to indicate the backup of the first file stored within the first object represents deleted data, in response to the deletion of the first file;
determining whether an access pattern associated with live data remaining within the first object satisfies a condition, the live data remaining within the first object including the backup of the second file; and
reallocating the live data remaining within the first object by creating a third object within the object storage to store the backup of the second file, in response to determining the access pattern associated with the live data satisfies the condition.
10. The method of claim 9 , further comprising:
determining whether an amount of the live data remaining within the first object falls below a predetermined threshold, wherein
reallocating the live data is further in response to determining the amount of the live data remaining within the first object falls below the predetermined threshold.
11. The method of claim 10 , wherein the predetermined threshold is based on an amount of the live data remaining within the first object compared to an amount of the deleted data within the first object.
12. The method of claim 9 , wherein the one or more conditions include:
the backup of the second file stored on the object storage being accessed a predetermined number of times within a given time period.
13. The method of claim 9 , wherein the one or more conditions include:
the second file stored on the client device being accessed a predetermined number of times within a given time period.
14. The method of claim 9 , wherein the one or more conditions include:
a predetermined amount of time elapsing since storing the backup of the second file on the object storage.
15. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
perform a first backup of client data stored on a client device at a first point-in-time to an object storage, the client data at the first point-in-time including a first file and a second file, the first backup including creating a first object within the object storage to store a backup of the first file and the second file;
perform, after the first backup, a second backup of the client data stored on the client device at a second point-in-time to the object storage, the client data at the second point-in-time including a new third file and a deletion of the first file, the second backup including creating a second object within the object storage to store a backup of the new third file;
update a metadata database associated with the object storage to indicate the backup of the first file stored within the first object represents deleted data, in response to the deletion of the first file;
determine whether an access pattern associated with live data remaining within the first object satisfies one or more conditions, the live data remaining within the first object including the backup of the second file; and
reallocate the live data remaining within the first object by creating a third object within the object storage to store the backup of the second file, in response to determining the access pattern associated with the live data satisfies the condition.
16. The computer program product of claim 15 , wherein the program code includes further instructions to:
determine whether an amount of the live data remaining within the first object falls below a predetermined threshold, wherein
reallocating the live data is further in response to determining the amount of the live data remaining within the first object falls below the predetermined threshold.
17. The computer program product of claim 16 , wherein the predetermined threshold is based on an amount of the live data remaining within the first object compared to an amount of the deleted data within the first object.
18. The computer program product of claim 15 , wherein the one or more conditions include:
the backup of the second file stored on the object storage being accessed a predetermined number of times within a given time period.
19. The computer program product of claim 15 , wherein the one or more conditions include:
the second file stored on the client device being accessed a predetermined number of times within a given time period.
20. The computer program product of claim 15 , wherein the one or more conditions include:
a predetermined amount of time elapsing since storing the backup of the second file on the object storage.