Efficiently managing point-in-time copies of data within a primary storage system
Described is a system for managing point-in-time backups (or “snapshots”) of data that are stored as part of a primary storage. As part of a data protection service, a set of snapshots may be stored on the primary storage system for efficient and immediate recovery, and a set of corresponding recovery snapshots may be stored on a secondary storage system for recovery purposes. Accordingly, the system may provide the ability to leverage the efficient storage mechanisms of the primary storage, while still maintaining the data storage efficiency (e.g. costs) of the secondary storage. Moreover, the system may manage the snapshots stored on the primary and secondary storages independently. For example, the system may fully leverage the available capacity of storage pools within the primary storage by automatically purging certain snapshots from the primary storage while still maintaining the corresponding recovery snapshots on the secondary storage.
1. A system comprising:
one or more processors; and
a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
store a set of point-in-time snapshots of data on a primary storage system, the data associated with a client of a data backup service;
store a recovery snapshot for each of the snapshots on a secondary storage system, the secondary storage system remote from the primary storage system;
create a most recent snapshot of the data and store the most recent snapshot on the primary storage system, in response to identifying a storage event associated with the data;
determine a limit for a total number of snapshots storable on the primary storage system has been reached based on a storage policy associated with the client;
delete an existing snapshot from the set of snapshots stored on the primary storage system based on an attribute associated with the existing snapshot, the deleting the existing snapshot from the set of snapshots includes retaining the recovery snapshot of the existing snapshot stored on the secondary storage system;
receive a request to recover the data to a particular point-in-time; and
recover the data by restoring the most recent snapshot directly from the primary storage system, in response to the particular point-in-time being associated with the storage event.
2. The system of claim 1 , wherein the deleting the existing snapshot from the set of snapshots stored on the primary storage system includes:
retrieving an expiration date attribute for each of the snapshots from the set of snapshots stored on the primary storage;
determining the existing snapshot has expired based on the expiration date attribute; and
deleting the existing snapshot from the primary storage system, in response to determining the existing snapshot has expired.
3. The system of claim 1 , wherein the deleting the existing snapshot from the set of snapshots stored on the primary storage system includes:
retrieving a creation time attribute for each of the snapshots from the set of snapshots stored on the primary storage;
determining the creation time attribute for the existing snapshot is the oldest creation time amongst the snapshots; and
deleting the existing snapshot from the primary storage system, in response to determining the creation time attribute for the existing snapshot is the oldest creation time amongst the snapshots.
4. The system of claim 1 , wherein determining the limit for the total number of snapshots storable on the primary storage system has been reached includes determining a limit for a number of snapshots associated with a particular LUN or consistency group has been reached.
5. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
recover the data by restoring one of the recovery snapshots stored on the secondary storage system, in response to the particular point-in-time being associated with the existing snapshot deleted from the primary storage system.
6. The system of claim 1 , wherein the data and the set of snapshots are stored as part of the same storage pool within the primary storage system.
7. The system of claim 1 , wherein recovering the data by restoring the most recent snapshot directly from the primary storage system includes immediately recovering the data by redirecting a set of pointers for snapshot data associated with the most recent snapshot.
8. A method comprising:
storing a set of point-in-time snapshots of data on a primary storage system, the data associated with a client of a data backup service;
storing a recovery snapshot for each of the snapshots on a secondary storage system, the secondary storage system remote from the primary storage system;
creating a most recent snapshot of the data and store the most recent snapshot on the primary storage system, in response to identifying a storage event associated with the data;
determining a limit for a total number of snapshots storable on the primary storage system has been reached based on a storage policy associated with the client;
deleting an existing snapshot from the set of snapshots stored on the primary storage system based on an attribute associated with the existing snapshot, the deleting the existing snapshot from the set of snapshots includes retaining the recovery snapshot of the existing snapshot stored on the secondary storage system;
receiving a request to recover the data to a particular point-in-time; and
recovering the data by restoring the most recent snapshot directly from the primary storage system, in response to the particular point-in-time being associated with the storage event.
9. The method of claim 8 , wherein the deleting the existing snapshot from the set of snapshots stored on the primary storage system includes:
retrieving an expiration date attribute for each of the snapshots from the set of snapshots stored on the primary storage;
determining the existing snapshot has expired based on the expiration date attribute; and
deleting the existing snapshot from the primary storage system, in response to determining the existing snapshot has expired.
10. The method of claim 8 , wherein the deleting the existing snapshot from the set of snapshots stored on the primary storage system includes:
retrieving a creation time attribute for each of the snapshots from the set of snapshots stored on the primary storage;
determining the creation time attribute for the existing snapshot is the oldest creation time amongst the snapshots; and
deleting the existing snapshot from the primary storage system, in response to determining the creation time attribute for the existing snapshot is the oldest creation time amongst the snapshots.
11. The method of claim 8 , wherein determining the limit for the total number of snapshots storable on the primary storage system has been reached includes determining a limit for a number of snapshots associated with a particular LUN or consistency group has been reached.
12. The method of claim 8 , further comprising:
recovering the data by restoring one of the recovery snapshots stored on the secondary storage system, in response to the particular point-in-time being associated with the existing snapshot deleted from the primary storage system.
13. The method of claim 8 , wherein the data and the set of snapshots are stored as part of the same storage pool within the primary storage system.
14. The method of claim 8 , wherein recovering the data by restoring the most recent snapshot directly from the primary storage system includes immediately recovering the data by redirecting a set of pointers for snapshot data associated with the most recent snapshot.
15. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
store a set of point-in-time snapshots of data on a primary storage system, the data associated with a client of a data backup service;
store a recovery snapshot for each of the snapshots on a secondary storage system, the secondary storage system remote from the primary storage system;
create a most recent snapshot of the data and store the most recent snapshot on the primary storage system, in response to identifying a storage event associated with the data;
determine a limit for a total number of snapshots storable on the primary storage system has been reached based on a storage policy associated with the client;
delete an existing snapshot from the set of snapshots stored on the primary storage system based on an attribute associated with the existing snapshot, the deleting the existing snapshot from the set of snapshots includes retaining the recovery snapshot of the existing snapshot stored on the secondary storage system;
receive a request to recover the data to a particular point-in-time; and
recover the data by restoring the most recent snapshot directly from the primary storage system, in response to the particular point-in-time being associated with the storage event.
16. The computer program product of claim 15 , wherein the deleting the existing snapshot from the set of snapshots stored on the primary storage system includes:
retrieving an expiration date attribute for each of the snapshots from the set of snapshots stored on the primary storage;
determining the existing snapshot has expired based on the expiration date attribute; and
deleting the existing snapshot from the primary storage system, in response to determining the existing snapshot has expired.
17. The computer program product of claim 15 , wherein the deleting the existing snapshot from the set of snapshots stored on the primary storage system includes:
retrieving a creation time attribute for each of the snapshots from the set of snapshots stored on the primary storage;
determining the creation time attribute for the existing snapshot is the oldest creation time amongst the snapshots; and
deleting the existing snapshot from the primary storage system, in response to determining the creation time attribute for the existing snapshot is the oldest creation time amongst the snapshots.
18. The computer program product of claim 15 , wherein determining the limit for the total number of snapshots storable on the primary storage system has been reached includes determining a limit for a number of snapshots associated with a particular LUN or consistency group has been reached.
19. The computer program product of claim 15 , wherein the program code includes further instructions to:
recover the data by restoring one of the recovery snapshots stored on the secondary storage system, in response to the particular point-in-time being associated with the existing snapshot deleted from the primary storage system.
20. The computer program product of claim 15 , wherein the data and the set of snapshots are stored as part of the same storage pool within the primary storage system, and wherein recovering the data by restoring the most recent snapshot directly from the primary storage system includes immediately recovering the data by redirecting a set of pointers for snapshot data associated with the most recent snapshot.