IP Library Granted Patent US 9,032,160
Granted Patent B1
US 9,032,160 · App. 13/339,780 · Granted May 12, 2015

Continuous data replication

Inventors: Assaf Natanzon (Tel Aviv, IL); Saar Cohen (Mishmeret, IL); Steven R. Bromling (Alberta, CA)
Assignee: EMC Corporation
G06F11/073G06F1/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,032,160
App. No.
13/339,780
Filed
Dec 29, 2011
Granted
May 12, 2015
Kind
B1
Art Unit
2132
USPC
711/135
Abstract

In a first embodiment, a method and computer program product for use in a storage system comprising quiescing IO commands the sites of an ACTIVE/ACTIVE storage system, the active/active storage system having at least two storage sites communicatively coupled via a virtualization layer, creating a change set, unquiescing IO commands by the virtualization layers, transferring data of a change set to the other sites of the active/active storage system by the virtualization layer, and flushing the data by the virtualization layer. In a second embodiment, a method and computer program product for use in a storage system comprising fracturing a cluster of an active/active storage system; wherein the cluster includes at least two sites, stopping IO on a first site of the cluster; and rolling to a point in time on the first site.

Claims (72)

1. A computer program product for use in replication comprising:

a non-transitory computer readable medium encoded with computer executable program code for replication of data, the code configured to enable the execution of:

quiescing IO commands to sites of an ACTIVE/ACTIVE storage system, the active/active storage system having at least two storage sites communicatively coupled via a virtualization layer; wherein the virtualization layer enables the at least two storage sites to present a same virtual volume at each of the at least two storage sites enabling transparent migration of a virtual machine between the at least two storage sites; wherein the virtualization layer enables cache coherency between each of the at least two storage sited; wherein the virtualization layer enables simultaneous, active/active, concurrent read/write access to the same virtual volume at both of the sites;

creating a change set for the same virtual volume;

unquiescing IO commands to the sites;

transferring data of the change set to the other sites of the active/active storage system by the virtualization layer, and

flushing the data to the sites; wherein flushing of the data of the change set enables the other sites of the at least two storage sites to synchronize changed data of the site from which the change set was transferred enabling the at least two storage sites to have a point in time with the same data.

2. The code of claim 1 wherein the flushing comprises:

reading, by the virtualization layer, an undo of the data to be flushed in the change set;

writing the undo data corresponding to the data in the write cache to a data protection appliance (DPA);

writing, by the DPA, the undo data to a journal; and

writing the data to the backend storage system by the virtualization layer.

3. The code of claim 2 where the DPA is a virtual machine; and the code further enables:

writing the data from the write cache to a volume;

notifying the DPA a full change set has been flushed; and

creating a bookmark at the undo journal by the DPA.

4. the code of claim 1 for use in replication enabling:

a non-transitory computer readable medium encoded with computer executable program code for replication of data, the code configured to enable the execution of:

fracturing a cluster of an active/active storage system; wherein the cluster includes at least two sites; wherein an active/active storage system that is not fractured includes a virtualization layer that enables the at least two storage sites to present a same virtual volume at each of the at least two storage sites enabling transparent migration of a virtual machine between the at least two storage sites; wherein the virtualization layer enables simultaneous, active/active, concurrent read/write access to the same virtual volume at both of the sites; wherein the virtualization layer enables cache coherency between each of the at least two storage sited; wherein a fractured active/active storage system enables an image on a first site of the sites to be rolled to a point in time and enables active production on a second site of the sites;

stopping IO on a first site of the cluster; and

rolling to a point in time on a first site of the cluster that is consistent with at least a second site of the cluster before the at least two sites were fractured.

5. The code of claim 4 further enabling:

flushing a write cache on a second site of the cluster to the first site of the cluster.

6. The code of claim 4 further enabling:

requesting, by a user, access to a point in time;

writing undo information generated by rolling to the point in time to a redo log;

exposing the point in time to the user;

enabling the user to access the point in time data; and

storing changes to the point in time in an undo stream.

7. The code of claim 4 further enabling:

determining to revert to the point in time; and

based on a positive determination stopping access to the second site, discarding the redo log on the first site, making the sites not fractured, synchronizing the sites to the point in time and allowing access to the second site.

8. The code of claim 7 further enabling:

based on a negative determination, stopping access to the first site discarding the changes made at point in time, rolling the first site to the latest point in time using the redo log, making the sites not fractured and synchronizing both sites to an image on the second site, and restoring access to the first site.

9. The code of claim 6 further enabling exposing the point in time to the user, where in the logical unit identity is different than the original logical unit identity.

10. The code of claim 9 further enabling: shutting down hosts on second site, before stopping access, and restarting hosts when access restored.

11. A computer implemented method use in replication, comprising:

quiescing IO commands to sites of an ACTIVE/ACTIVE storage system, the active/active storage system having at least two storage sites communicatively coupled via a virtualization layer; wherein the virtualization layer virtual service layer enables the at least two storage sites to present a same virtual volume at each of the at least two storage sites enabling transparent migration of a virtual machine between the at least two storage sites; wherein the virtualization layer enables simultaneous, active/active, concurrent read/write access to the same virtual volume at both of the sites; wherein the virtualization layer enables cache coherency between each of the at least two storage sited;

creating a change set for the same virtual volume;

unquiescing IO commands to the sites;

transferring data of a change set to the other sites of the active/active storage system by the virtualization layer, and

flushing the data; wherein flushing of the data of the change set enables the other sites of the at least two storage sites to synchronize changed data of the site from which the change set was transferred enabling the at least two storage sites to have a point in time with the same data.

12. The method of claim 11 further comprising:

reading, by the virtualization layer, the undo of the data to be flushed,

writing undo data corresponding to the data in the write cache to a data protection appliance (DPA);

writing, by the DPA, the undo data to a journal; and

writing the data to the backend storage system by the virtualization layer.

13. The method of claim 11 where the DPA is virtual machine and the method further comprising:

writing the data from the write cache to a volume;

notifying the DPA a full change set has been flushed; and

creating a bookmark at the undo journal by the DPA.

14. the code of claim 4 further comprising:

fracturing a cluster of an active/active storage system; wherein the cluster includes at least two sites; wherein an active/active storage system that is not fractured includes a virtualization layer enables the at least two storage sites to present a same virtual volume at each of the at least two storage sites enabling transparent migration of a virtual machine between the at least two storage sites; wherein the virtualization layer enables simultaneous, active/active, concurrent read/write access to the same virtual volume at both of the sites; wherein the virtualization layer enables cache coherency between each of the at least two storage sited; wherein a fractured active/active storage system enables an image on a first site of the sites to be rolled to a point in time and enables active production on a second site of the sites;

stopping IO on a first site of the cluster; and

rolling to a point in time on a first site of the cluster that is consistent with at least a second site of the cluster before the at least two sites were fractured.

15. The computer implemented method of claim 14 further comprising:

flushing the a write cache on a second site of the cluster to the first site of the cluster.

16. The computer implemented method of claim 14 further comprising:

requesting, by a user, access to a point in time;

writing undo information generated by rolling to the point in time to a redo log;

exposing the point in time to the user;

enabling the user to access the point in time data; and

storing changes to the point in time in an undo stream.

17. The computer implemented method of claim 14 further comprising:

determining to revert to the point in time; and

based on a positive determination stopping access to the second site, discarding the redo log on the first site, un-fracturing the sites by unfracturing the virtualization layer, synchronizing sites to the point in time and allowing access to the second site.

18. The computer implemented method of claim 17 further comprising:

based on a negative determination, stopping access to the first site discarding changes made at the point in time, rolling the first site to the latest point in time using the redo log, making the sites not fractured by connecting the virtualization layer and synchronizing both sites to an image on the second site, and restoring access to the first site.

19. The computer implemented method of claim 16 further comprising:

exposing the point in time to the user, where in the logical unit identity is different than the original logical unit identity.

20. The computer implemented method of claim 14 further comprising:

shutting down hosts on second site, before stopping access, and restarting hosts when access restored.

Assignments (12)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2013
From: EMC INTERNATIONAL COMPANY
To: EMC CORPORATION
Reel/Frame 030847/0412 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 24, 2013
From: NATANZON, ASSAF; COHEN, SAAR
To: EMC INTERNATIONAL COMPANY
Reel/Frame 030480/0225 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 24, 2013
From: BROMLING, STEVEN R
To: EMC CORPORATION
Reel/Frame 030480/0214 →