IP Library Granted Patent US 12,346,206
Granted Patent B2
US 12,346,206 · App. 16/697,544 · Granted Jul 1, 2025

Incremental snapshot recovery from remote storage system

Inventors: Deepak Kumar Shivanna Vokaliga (Hopkinton, MA); Michael Daniel Anthony (Wilmington, MA)
Assignee: EMC IP Holding Company LLC
G06F11/1451G06F2201/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,346,206
App. No.
16/697,544
Granted
Jul 1, 2025
Kind
B2
Abstract

Techniques are provided for incremental snapshot recovery from a remote storage system. One method comprises, in response to a request to restore a snapshot from a remote storage system: obtaining, from a local storage system, a first bitmap for the first snapshot indicating differences with respect to a previous snapshot, comparing the first bitmap to an allocation bitmap of an earliest maintained snapshot to obtain a baseline bitmap representing a baseline version of the first snapshot; recovering the baseline version of the first snapshot, using the baseline bitmap, from the remote storage system. The snapshots may be stored in the remote storage system in a time order, and following the recovery of the first snapshot, a user may move forward or backward in time along a lineage of snapshots by copying only the changed blocks of bitmaps between two desired points in time.

Claims (53)

1. A method, comprising:

in response to a request to restore a first snapshot of a logical unit from a remote storage system to a local storage system, wherein the local storage system is remote from the remote storage system:

obtaining, from the local storage system, a first bitmap for the first snapshot to be restored, wherein the first bitmap indicates differences with respect to a previous snapshot of the logical unit, wherein an earliest maintained snapshot for the logical unit comprises a full snapshot having an allocation bitmap, wherein the allocation bitmap was created prior to receiving the request to restore the first snapshot;

comparing, by at least one processing device associated with the local storage system, the first bitmap to the allocation bitmap associated with the full snapshot to obtain a baseline bitmap representing a baseline version of the first snapshot to be restored;

recovering, by the at least one processing device associated with the local storage system, the baseline version of the first snapshot to be restored, using the baseline bitmap, from the remote storage system to the local storage system, wherein a plurality of snapshots stored in the remote storage system are stored in a time order and wherein the plurality of snapshots stored in the time order are configured to be traversed in a forward direction and a backward direction in time;

storing the baseline version of the first snapshot to be restored as a last recovered snapshot with the baseline bitmap to a recovery logical unit in the local storage system; and

following the storing the baseline version of the first snapshot in the recovery logical unit in the local storage system, (i) traversing at least one of forward and backward in time along a lineage of snapshots to recover a second snapshot from the remote storage system to the local storage system, wherein the second snapshot is non-consecutive with the first snapshot, by determining one or more changed blocks of the second snapshot, relative to the first non-consecutive snapshot, wherein the one or more changed blocks of the second snapshot are determined by applying an aggregation function to a plurality of bitmaps between the first snapshot and the second snapshot, and wherein the plurality of bitmaps between the first snapshot and the second snapshot comprise a plurality of bits identifying differences with respect to a corresponding snapshot, (ii) performing a differential recovery of the second snapshot from the remote storage system to the local storage system by: (a) copying the determined one or more changed blocks of the second snapshot, relative to the first non-consecutive snapshot, from the remote storage system and (b) using the first snapshot in the local storage system, and (iii) storing the recovered second snapshot in the recovery logical unit in the local storage system;

wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein the comparing comprises performing a bitwise OR operation between the first bitmap and the allocation bitmap.

3. The method of claim 1 , further comprising, in response to a request to restore a second snapshot of the logical unit from the remote storage system, wherein a second bitmap for the second snapshot indicates differences with respect to a previous snapshot of the logical unit:

obtaining the last recovered snapshot and the baseline bitmap from the local storage system;

comparing the second bitmap for the second snapshot to the baseline bitmap for the last recovered snapshot to identify changes to the second snapshot relative to the last recovered snapshot;

performing a differential recovery of the second snapshot, using the identified changes and the last recovered snapshot, from the remote storage system; and

storing the second snapshot as the last recovered snapshot as a new baseline version to the recovery logical unit in the local storage system.

4. The method of claim 1 , wherein a snapshotting module in the local storage system generates the first snapshot according to a predefined data retention policy and wherein an incremental storage module transfers an incremental representation of the first snapshot, relative to the previous snapshot, to the remote storage system.

5. The method of claim 1 , wherein at least a portion of the remote storage system comprises a cloud-based storage system.

6. The method of claim 3 , wherein the comparing comprises performing a bitwise OR operation between the second bitmap and the baseline bitmap.

7. The method of claim 3 , wherein, when the second snapshot is after the last recovered snapshot, the differential recovery of the second snapshot comprises reversing one or more write operations associated with the last recovered snapshot.

8. The method of claim 7 , wherein the reversing the one or more write operations associated with the last recovered snapshot comprises performing bitwise OR operations between each subsequent pair of bitmaps between the last recovered snapshot and the second snapshot to obtain a representation of the second snapshot and then performing one or more read operations from the second snapshot from the remote storage system to the recovery logical unit.

9. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

the at least one processing device being configured to implement the following steps:

in response to a request to restore a first snapshot of a logical unit from a remote storage system to a local storage system, wherein the local storage system is remote from the remote storage system:

obtaining, from the local storage system, a first bitmap for the first snapshot to be restored, wherein the first bitmap indicates differences with respect to a previous snapshot of the logical unit, wherein an earliest maintained snapshot for the logical unit comprises a full snapshot having an allocation bitmap, wherein the allocation bitmap was created prior to receiving the request to restore the first snapshot;

comparing, by at least one processing device associated with the local storage system, the first bitmap to the allocation bitmap associated with the full snapshot to obtain a baseline bitmap representing a baseline version of the first snapshot to be restored;

recovering, by the at least one processing device associated with the local storage system, the baseline version of the first snapshot to be restored, using the baseline bitmap, from the remote storage system to the local storage system, wherein a plurality of snapshots stored in the remote storage system are stored in a time order and wherein the plurality of snapshots stored in the time order are configured to be traversed in a forward direction and a backward direction in time;

storing the baseline version of the first snapshot to be restored as a last recovered snapshot with the baseline bitmap to a recovery logical unit in the local storage system; and

following the storing the baseline version of the first snapshot in the recovery logical unit in the local storage system, (i) traversing at least one of forward and backward in time along a lineage of snapshots to recover a second snapshot from the remote storage system to the local storage system, wherein the second snapshot is non-consecutive with the first snapshot, by determining one or more changed blocks of the second snapshot, relative to the first non-consecutive snapshot, wherein the one or more changed blocks of the second snapshot are determined by applying an aggregation function to a plurality of bitmaps between the first snapshot and the second snapshot, and wherein the plurality of bitmaps between the first snapshot and the second snapshot comprise a plurality of bits identifying differences with respect to a corresponding snapshot, (ii) performing a differential recovery of the second snapshot from the remote storage system to the local storage system by: (a) copying the determined one or more changed blocks of the second snapshot, relative to the first non-consecutive snapshot, from the remote storage system and (b) using the first snapshot in the local storage system, and (iii) storing the recovered second snapshot in the recovery logical unit in the local storage system.

10. The apparatus of claim 9 , wherein, when the second snapshot is after the last recovered snapshot, the differential recovery of the second snapshot comprises reversing one or more write operations associated with the last recovered snapshot.

11. The apparatus of claim 9 , wherein a snapshotting module in the local storage system generates the first snapshot according to a predefined data retention policy and wherein an incremental storage module transfers an incremental representation of the first snapshot, relative to the previous snapshot, to the remote storage system.

12. The apparatus of claim 9 , wherein the comparing comprises performing a bitwise OR operation between the first bitmap and the allocation bitmap.

13. The apparatus of claim 9 , further comprising, in response to a request to restore a second snapshot of the logical unit from the remote storage system, wherein a second bitmap for the second snapshot indicates differences with respect to a previous snapshot of the logical unit:

obtaining the last recovered snapshot and the baseline bitmap from the local storage system;

comparing the second bitmap for the second snapshot to the baseline bitmap for the last recovered snapshot to identify changes to the second snapshot relative to the last recovered snapshot;

performing a differential recovery of the second snapshot, using the identified changes and the last recovered snapshot, from the remote storage system; and

storing the second snapshot as the last recovered snapshot as a new baseline version to the recovery logical unit in the local storage system.

14. The apparatus of claim 10 , wherein the reversing the one or more write operations associated with the last recovered snapshot comprises performing bitwise OR operations between each subsequent pair of bitmaps between the last recovered snapshot and the second snapshot to obtain a representation of the second snapshot and then performing one or more read operations from the second snapshot from the remote storage system to the recovery logical unit.

15. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps:

in response to a request to restore a first snapshot of a logical unit from a remote storage system to a local storage system, wherein the local storage system is remote from the remote storage system:

obtaining, from the local storage system, a first bitmap for the first snapshot to be restored, wherein the first bitmap indicates differences with respect to a previous snapshot of the logical unit, wherein an earliest maintained snapshot for the logical unit comprises a full snapshot having an allocation bitmap, wherein the allocation bitmap was created prior to receiving the request to restore the first snapshot;

comparing, by at least one processing device associated with the local storage system, the first bitmap to the allocation bitmap associated with the full snapshot to obtain a baseline bitmap representing a baseline version of the first snapshot to be restored;

recovering, by the at least one processing device associated with the local storage system, the baseline version of the first snapshot to be restored, using the baseline bitmap, from the remote storage system to the local storage system, wherein a plurality of snapshots stored in the remote storage system are stored in a time order and wherein the plurality of snapshots stored in the time order are configured to be traversed in a forward direction and a backward direction in time;

storing the baseline version of the first snapshot to be restored as a last recovered snapshot with the baseline bitmap to a recovery logical unit in the local storage system; and

following the storing the baseline version of the first snapshot in the recovery logical unit in the local storage system, (i) traversing at least one of forward and backward in time along a lineage of snapshots to recover a second snapshot from the remote storage system to the local storage system, wherein the second snapshot is non-consecutive with the first snapshot, by determining one or more changed blocks of the second snapshot, relative to the first non-consecutive snapshot, wherein the one or more changed blocks of the second snapshot are determined by applying an aggregation function to a plurality of bitmaps between the first snapshot and the second snapshot, and wherein the plurality of bitmaps between the first snapshot and the second snapshot comprise a plurality of bits identifying differences with respect to a corresponding snapshot, (ii) performing a differential recovery of the second snapshot from the remote storage system to the local storage system by: (a) copying the determined one or more changed blocks of the second snapshot, relative to the first non-consecutive snapshot, from the remote storage system and (b) using the first snapshot in the local storage system, and (iii) storing the recovered second snapshot in the recovery logical unit in the local storage system.

16. The non-transitory processor-readable storage medium of claim 15 , wherein the comparing comprises performing a bitwise OR operation between the first bitmap and the allocation bitmap.

17. The non-transitory processor-readable storage medium of claim 15 , further comprising, in response to a request to restore a second snapshot of the logical unit from the remote storage system, wherein a second bitmap for the second snapshot indicates differences with respect to a previous snapshot of the logical unit:

obtaining the last recovered snapshot and the baseline bitmap from the local storage system;

comparing the second bitmap for the second snapshot to the baseline bitmap for the last recovered snapshot to identify changes to the second snapshot relative to the last recovered snapshot;

performing a differential recovery of the second snapshot, using the identified changes and the last recovered snapshot, from the remote storage system; and

storing the second snapshot as the last recovered snapshot as a new baseline version to the recovery logical unit in the local storage system.

18. The non-transitory processor-readable storage medium of claim 15 , wherein a snapshotting module in the local storage system generates the first snapshot according to a predefined data retention policy and wherein an incremental storage module transfers an incremental representation of the first snapshot, relative to the previous snapshot, to the remote storage system.

19. The non-transitory processor-readable storage medium of claim 17 , wherein, when the second snapshot is after the last recovered snapshot, the differential recovery of the second snapshot comprises reversing one or more write operations associated with the last recovered snapshot.

20. The non-transitory processor-readable storage medium of claim 19 , wherein the reversing the one or more write operations associated with the last recovered snapshot comprises performing bitwise OR operations between each subsequent pair of bitmaps between the last recovered snapshot and the second snapshot to obtain a representation of the second snapshot and then performing one or more read operations from the second snapshot from the remote storage system to the recovery logical unit.

Assignments (9)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053311/0169) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 060438/0742 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052216/0758) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060438/0680 →
RELEASE OF SECURITY INTEREST AF REEL 052243 FRAME 0773 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0152 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 053311/0169 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 26, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 052243/0773 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Mar 24, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052216/0758 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2019
From: VOKALIGA, DEEPAK KUMAR SHIVANNA; ANTHONY, MICHAEL DANIEL
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 051128/0173 →