IP Library › Granted Patent US 10,235,088
Granted Patent B1
US 10,235,088 · App. 15/085,142 · Granted Mar 19, 2019

Global replication policy for multi-copy replication

Inventors: Leehod Baruch (Rishon Leziyon, IL); Assaf Natanzon (Tel Aviv, IL); Jehuda Shemer (Kfar Saba, IL); Saar Cohen (Moshav Mishmeret, IL); Amit Lieberman (Raanana, IL)
Assignee: EMC IP Holding Company LLC
G06F3/065G06F3/0619G06F3/0683
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,235,088
App. No.
15/085,142
Filed
Mar 30, 2016
Granted
Mar 19, 2019
Kind
B1
Art Unit
2132
USPC
711/162
Abstract

In one embodiment, a method includes measuring a first parameter associated with copying a storage entity to a first backup site, measuring a second parameter associated with copying the storage entity to a second backup site and determining a replication mode to copy the storage entity to the first backup site and a replication mode to copy the storage entity to the second backup site in response to a replication policy and the first and second parameters measured.

Claims (38)

1. A method comprising: performing replication, in accordance with a replication policy, of storage entities from a source site to a first backup site over a first link and from the source site to a second backup site over a second link;

after a first time period:

measuring a first parameter associated with copying a first storage entity of the storage entities to the first backup site over the first link;

measuring the parameter associated with copying a second storage entity of the storage entities to the second backup site over the second link;

comparing values of the parameter derived from the measuring over the first and second links;

and dynamically changing a replication mode to copy the first storage entity to the first backup site and a replication mode to copy the second storage entity to the second backup site as a function of the replication policy and the measured values of the parameter, the changing configured to compensate for variances in the relative values of the parameter measured,

wherein the replication mode to copy the first storage entity is a first type of replication mode and the replication mode to copy the second storage entity is a second type of replication mode that is different than the first type of replication mode, and wherein the dynamically changing includes switching the replication mode of the first storage entity to the second type of replication mode and switching the replication mode of the second storage entity to the first type of replication mode;

wherein the measuring the parameter is a second measurement of the parameter at a time succeeding the first time period, wherein when the parameter measured is latency, and upon determining at a time preceding the first time period, the measured parameter indicates that latency of data transfer is smaller on the first link than on the second link, and upon determining that the replication mode to copy the storage entity to the first backup site is synchronous replication and that the replication mode to copy the first storage entity to the second backup site is asynchronous replication, performing the synchronous and asynchronous replications prior to conclusion of the time period; wherein, after the first time period, and in response to performing second measurements of the parameter, determining the measured parameter indicates that the latency of data transfer is larger on the first link than on the second link, changing the replication mode to copy the first storage entity to the first backup site from synchronous replication to asynchronous replication and changing the replication mode to copy the second storage entity to the second backup site from asynchronous replication to synchronous replication.

2. The method of claim 1 , wherein the parameter comprises cost of storage on the first and second backup sites, and

wherein the dynamically adjusting the replication mode to copy the first storage entity to the first back site and the replication mode to copy the second storage entity to the second backup site includes changing the replication mode to snapshot replication mode for one of the storage entities corresponding to one of the first and second backup sites in which capacity of the respective backup site is determined to be expensive for corresponding one of the storage entities relative to another of the back up sites.

3. The method of claim 1 , further comprising storing the replication policy, the replication policy comprising at least one of a number of copies of the storage entity required, backup sites to store the copies of the storage entity, a number of links to use for synchronous, asynchronous or snapchat-based replication, and a time interval to use each replication mode.

4. The method of claim 1 , wherein the storage entities are consistency groups comprising logical units.

5. An apparatus, comprising:

electronic hardware circuitry configured to: perform replication, in accordance with a replication policy, of storage entities from a source site to a first backup site over a first link and from the source site to a second backup site over a second link;

after a first time period:

measure a parameter associated with copying a first storage entity of the storage entities to the first backup site over the first link;

measure the parameter associated with copying a second the storage entity of the storage entities to the second backup site over the second link;

compare values of the parameter derived from the measuring over the first and second links;

and dynamically change a replication mode to copy the first storage entity to the first backup site and a replication mode to copy the second storage entity to the second backup site as a function of the replication policy and the first and second parameters measured values of the parameter, the changes configured to compensate for variances in the relative values of the parameter measured;

wherein the measuring the parameter is a second measurement of the parameter at a time succeeding the first time period, wherein when the parameter measured is latency, and upon determining at a time preceding the first time period, the measured parameter indicates that latency of data transfer is smaller on the first link than on the second link, and upon determining that the replication mode to copy the storage entity to the first backup site is synchronous replication and that the replication mode to copy the first storage entity to the second backup site is asynchronous replication, performing the synchronous and asynchronous replications prior to conclusion of the time period; wherein, after the first time period, and in response to performing second measurements of the parameter, determining the measured parameter indicates that the latency of data transfer is larger on the first link than on the second link, changing the replication mode to copy the first storage entity to the first backup site from synchronous replication to asynchronous replication and changing the replication mode to copy the second storage entity to the second backup site from asynchronous replication to synchronous replication.

6. The apparatus of claim 5 , wherein the circuitry comprises at least one of a processor, a memory, a programmable logic device of a logic gate.

7. The apparatus of claim 5 , wherein the group of parameters further comprise cost of storage on the first and second backup sites.

8. The apparatus of claim 5 , further comprising circuitry configured to store the replication policy, the replication policy comprising at least one of a number of copies of the storage entity required, backup sites to store the copies of the storage entity, a number of links to use for synchronous, asynchronous or snapchat-based replication, and time interval to use each replication mode.

9. The apparatus of claim 5 , wherein the storage entities are consistency groups comprising logical units.

10. An article comprising: a non-transitory computer-readable medium that stores computer-executable instructions, the instructions causing a machine to: perform replication, in accordance with a replication policy, of storage entities from a source site to a first backup site over a first link and from the source site to a second backup site over a second link;

after a first time period:

measure a first parameter associated with copying a first storage entity of the storage entities to a first backup site over the first link;

measure the second parameter associated with copying a second the storage entity of the storage entities to the second backup site over the second link;

compare values derived from the measuring over the first and second links;

and dynamically change a replication mode to copy the first storage entity to the first backup site and a replication mode to copy the second storage entity to the second backup site in response to as a function of the replication policy and the first and second parameters measured values of the parameter, the changes configured to compensate for variances in the relative values of the parameter measured;

wherein the measuring the parameter is a second measurement of the parameter at a time succeeding the first time period, wherein when the parameter measured is latency, and upon determining at a time preceding the first time period, the measured parameter indicates that latency of data transfer is smaller on the first link than on the second link, and upon determining that the replication mode to copy the storage entity to the first backup site is synchronous replication and that the replication mode to copy the first storage entity to the second backup site is asynchronous replication, performing the synchronous and asynchronous replications prior to conclusion of the time period; wherein, after the first time period, and in response to performing second measurements of the parameter, determining the measured parameter indicates that the latency of data transfer is larger on the first link than on the second link, changing the replication mode to copy the first storage entity to the first backup site from synchronous replication to asynchronous replication and changing the replication mode to copy the second storage entity to the second backup site from asynchronous replication to synchronous replication.

11. The article of claim 10 , wherein the parameters further comprise a cost of storage on the first and second backup sites.

12. The article of claim 10 , further comprising instructions causing the machine to store the replication policy, the replication policy comprising at least one of a number of copies of the storage entity required, backup sites to store the copies of the storage entity, a number of links to use for synchronous, asynchronous or snapshot-based replication, and a time interval to use each replication mode.

13. The article of claim 10 , wherein the storage entities are consistency groups comprising logical units.

14. The method of claim 1 , wherein when the parameter measured is bandwidth, the storage entities are consistency groups (CGs), and the mode of replication for a first portion of the CGs on the first link is synchronous replication mode, the mode of replication for a second portion of the CGs on the first link is asynchronous replication mode, the mode of replication for a first portion of the CGs on the CGs on the second link is synchronous replication mode, and the mode of replication for a second portion of the CGs on the second link is asynchronous replication mode,

wherein upon determining the measured parameter indicates that the bandwidth is smaller on the first link than on the second link, selecting at least one of the CGs from the second portion of the CGs operating in the asynchronous replication mode and changing the replication mode for the at least on CG to synchronous replication mode, and selecting at least one of the CGs from the first portion of the CGs on the first link operating in synchronous replication mode and changing the replication mode for the at least one CG of the first portion of the CGs to asynchronous replication mode.

15. The method of claim 1 , wherein the parameter measured is bandwidth, and the dynamically changing the replication mode is based on inter-consistency groups (CGs), the changing comprising selecting one of the first and second links having higher bandwidth for the CGs to use a continuous replication mode and selecting another of the first and second links having limited amount of bandwidth to use a snapshot-based replication mode.

16. The method of claim 1 , wherein the links comprise a wide area network.

Assignments (11)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (050724/0466) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO WYSE TECHNOLOGY L.L.C.)
Reel/Frame 060753/0486 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Oct 15, 2019
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 050724/0466 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2016
From: BARUCH, LEEHOD; NATANZON, ASSAF; SHEMER, JEHUDA; COHEN, SAAR; LIEBERMAN, AMIT
To: EMC CORPORATION
Reel/Frame 038207/0982 →
Cited By (1)
US 12,664,056