IP Library Granted Patent US 10,992,768
Granted Patent B1
US 10,992,768 · App. 16/824,988 · Granted Apr 27, 2021

Resuming copying of snapshots from a storage system to cloud storage

Inventors: Anurag Bhatnagar (Bangalore, IN); Michael Anthony (Wilmington, MA); Meenakshi Rani (Bathinda, IN); Shanmuga Anand Gunasekaran (Bangalore, IN)
Assignee: EMC IP Holding Company LLC
H04L67/2842
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,992,768
App. No.
16/824,988
Granted
Apr 27, 2021
Kind
B1
Abstract

An apparatus comprises a processing device configured to detect a request to resume copying of a snapshot of a storage volume from a storage system to cloud storage, to determine an original snapshot differential for the snapshot comprising differential data of the storage volume on the storage system, and to obtain checkpointing information for the snapshot characterizing a status of copying portions of the snapshot to the cloud storage. The processing device is also configured to generate a resume snapshot differential for the snapshot comprising portions of the snapshot to be copied to the cloud storage determined based at least in part on a comparison of the original snapshot differential with the checkpointing information characterizing the status of copying the one or more portions of the snapshot to the cloud storage. The processing device is further configured to resume copying of the snapshot utilizing the resume snapshot differential.

Claims (48)

1. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

the at least one processing device being configured:

to detect a request to resume copying of at least one snapshot of a storage volume from a storage system to cloud storage of at least one cloud external to the storage system;

to determine an original snapshot differential for the at least one snapshot, the original snapshot differential comprising differential data of the storage volume on the storage system that forms the at least one snapshot;

to obtain checkpointing information for the at least one snapshot from a checkpointing cache associated with the at least one processing device, the checkpointing information characterizing a status of copying one or more portions of the at least one snapshot to the cloud storage;

to generate a resume snapshot differential for the at least one snapshot, the resume snapshot differential comprising at least one portion of the at least one snapshot that is to be copied to the cloud storage, the at least one portion of the at least one snapshot that is to be copied to the cloud storage being determined based at least in part on a comparison of the original snapshot differential with the checkpointing information characterizing the status of copying the one or more portions of the at least one snapshot to the cloud storage; and

to resume copying of the at least one snapshot from the storage system to the cloud storage utilizing the resume snapshot differential.

2. The apparatus of claim 1 wherein detecting the request to resume copying of the at least one snapshot from the storage system to the cloud storage comprises obtaining, from the cloud storage, information characterizing one or more cloud object parts stored in the cloud storage for the at least one snapshot.

3. The apparatus of claim 2 wherein the information characterizing the one or more cloud object parts stored in the cloud storage for the at least one snapshot comprises, for a given one of the one or more cloud object parts, a size of the given cloud object part.

4. The apparatus of claim 3 wherein the checkpointing information characterizing the status of copying the one or more portions of the at least one snapshot to the cloud storage comprises, for a given one of the one or more portions of the at least one snapshot, a size of the given portion of the at least one snapshot.

5. The apparatus of claim 4 wherein generating the resume snapshot differential comprises comparing the size of the given cloud object part and the size of the given portion of the at least one snapshot and, responsive to determining that the size of the given cloud object part is not the same as the size of the given portion of the at least one snapshot, removing the given cloud object part from the cloud storage and adding the given portion of the at least one snapshot to the resume snapshot differential.

6. The apparatus of claim 4 wherein generating the resume snapshot differential comprises comparing the size of the given cloud object part and the size of the given portion of the at least one snapshot and, responsive to determining that the size of the given cloud object part is the same as the size of the given portion of the at least one snapshot, updating the checkpointing information in the checkpointing cache to indicate that the given portion of the at least one snapshot has been successfully copied to the cloud storage and removing the given portion of the at least one snapshot from the resume snapshot differential.

7. The apparatus of claim 1 wherein the at least one snapshot is copied from the storage system to the cloud storage utilizing at least one multi-part upload associated with at least one upload identifier, and wherein detecting the request to resume copying of the at least one snapshot from the storage system to the cloud storage comprises obtaining a multi-part upload list from the cloud storage, the multi-part upload list indicating at least a subset of the portions of the at least one snapshot that have been copied to the cloud storage.

8. The apparatus of claim 1 wherein the at least one processing device is further configured, prior to generating the resume snapshot differential, to validate a state of the checkpointing information in the checkpointing cache.

9. The apparatus of claim 8 wherein validating the state of the checkpointing information in the checkpointing cache comprises, for a given portion of data of the at least one snapshot:

determining whether the given portion of the data of the at least one snapshot is at least one of cached in the checkpointing cache, uploaded to the cloud storage, and committed to the cloud storage; and

determining whether metadata associated with the given portion of the data of the at least one snapshot is at least one of cached in the checkpointing cache, uploaded to the cloud storage, and committed to the cloud storage.

10. The apparatus of claim 1 wherein the at least one processing device is further configured to store, in the checkpointing cache, at least a portion of data of the at least one snapshot copied from the storage system.

11. The apparatus of claim 10 wherein copying of the at least one snapshot from the storage system to the cloud storage comprises compressing the differential data of the storage volume, and wherein the portion of the data of the at least one snapshot stored in the checkpointing cache comprises compressed data of the at least one snapshot.

12. The apparatus of claim 10 wherein copying of the at least one snapshot from the storage system to the cloud storage comprises encrypting the differential data of the storage volume, and wherein the portion of the data of the at least one snapshot stored in the checkpointing cache comprises encrypted data of the at least one snapshot.

13. The apparatus of claim 1 wherein the storage volume comprises one of:

at least one of one or more storage devices of the storage system;

a given logical unit provided by at least one of the one or more storage devices;

a consistency group comprising a set of two or more logical units provided by at least one of the one or more storage devices; and

an access-restricted storage group comprising a set of two or more logical units provided by at least one of the one or more storage devices where access to the storage group is limited to one or more designated host devices.

14. The apparatus of claim 1 wherein the at least one processing device is implemented external to the storage system.

15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform steps of:

detecting a request to resume copying of at least one snapshot of a storage volume from a storage system to cloud storage of at least one cloud external to the storage system;

determining an original snapshot differential for the at least one snapshot, the original snapshot differential comprising differential data of the storage volume on the storage system that forms the at least one snapshot;

obtaining checkpointing information for the at least one snapshot from a checkpointing cache associated with the at least one processing device, the checkpointing information characterizing a status of copying one or more portions of the at least one snapshot to the cloud storage;

generating a resume snapshot differential for the at least one snapshot, the resume snapshot differential comprising at least one portion of the at least one snapshot that is to be copied to the cloud storage, the at least one portion of the at least one snapshot that is to be copied to the cloud storage being determined based at least in part on a comparison of the original snapshot differential with the checkpointing information characterizing the status of copying the one or more portions of the at least one snapshot to the cloud storage; and

resuming copying of the at least one snapshot from the storage system to the cloud storage utilizing the resume snapshot differential.

16. The computer program product of claim 15 wherein the at least one snapshot is copied from the storage system to the cloud storage utilizing at least one multi-part upload associated with at least one upload identifier, and wherein detecting the request to resume copying of the at least one snapshot from the storage system to the cloud storage comprises obtaining a multi-part upload list from the cloud storage, the multi-part upload list indicating at least a subset of the portions of the at least one snapshot that have been copied to the cloud storage.

17. The computer program product of claim 15 wherein the program code when executed further causes the at least one processing device, prior to generating the resume snapshot differential, to perform the step of validating a state of the checkpointing information in the checkpointing cache, wherein validating the state of the checkpointing information in the checkpointing cache comprises, for a given portion of data of the at least one snapshot:

determining whether the given portion of the data of the at least one snapshot is at least one of cached in the checkpointing cache, uploaded to the cloud storage, and committed to the cloud storage; and

determining whether metadata associated with the given portion of the data of the at least one snapshot is at least one of cached in the checkpointing cache, uploaded to the cloud storage, and committed to the cloud storage.

18. A method comprising steps of:

detecting a request to resume copying of at least one snapshot of a storage volume from a storage system to cloud storage of at least one cloud external to the storage system;

determining an original snapshot differential for the at least one snapshot, the original snapshot differential comprising differential data of the storage volume on the storage system that forms the at least one snapshot;

obtaining checkpointing information for the at least one snapshot from a checkpointing cache associated with at least one processing device, the checkpointing information characterizing a status of copying one or more portions of the at least one snapshot to the cloud storage;

generating a resume snapshot differential for the at least one snapshot, the resume snapshot differential comprising at least one portion of the at least one snapshot that is to be copied to the cloud storage, the at least one portion of the at least one snapshot that is to be copied to the cloud storage being determined based at least in part on a comparison of the original snapshot differential with the checkpointing information characterizing the status of copying the one or more portions of the at least one snapshot to the cloud storage; and

resuming copying of the at least one snapshot from the storage system to the cloud storage utilizing the resume snapshot differential;

wherein the method is performed by the at least one processing device, the at least one processing device comprising a processor coupled to a memory.

19. The method of claim 18 wherein the at least one snapshot is copied from the storage system to the cloud storage utilizing at least one multi-part upload associated with at least one upload identifier, and wherein detecting the request to resume copying of the at least one snapshot from the storage system to the cloud storage comprises obtaining a multi-part upload list from the cloud storage, the multi-part upload list indicating at least a subset of the portions of the at least one snapshot that have been copied to the cloud storage.

20. The method of claim 18 further comprising, prior to generating the resume snapshot differential, validating a state of the checkpointing information in the checkpointing cache, wherein validating the state of the checkpointing information in the checkpointing cache comprises, for a given portion of data of the at least one snapshot:

determining whether the given portion of the data of the at least one snapshot is at least one of cached in the checkpointing cache, uploaded to the cloud storage, and committed to the cloud storage; and

determining whether metadata associated with the given portion of the data of the at least one snapshot is at least one of cached in the checkpointing cache, uploaded to the cloud storage, and committed to the cloud storage.

Assignments (13)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052851/0081) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0441 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052851/0917) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0509 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052852/0022) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0582 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053311/0169) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 060438/0742 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST AT REEL 052771 FRAME 0906 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0298 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052851/0081 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052851/0917 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052852/0022 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 053311/0169 →
SECURITY AGREEMENT Recorded May 28, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 052771/0906 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2020
From: BHATNAGAR, ANURAG; ANTHONY, MICHAEL; RANI, MEENAKSHI; GUNASEKARAN, SHANMUGA ANAND
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 052176/0019 →
Cited By (4)
US 12,242,418 US 12,259,791 US 12,632,340 US 12,664,089