IP Library Granted Patent US 9,619,264
Granted Patent B1
US 9,619,264 · App. 13/599,235 · Granted Apr 11, 2017

AntiAfinity

Inventors: Assaf Natanzon (Tel Aviv, IL); Saar Cohen (Mishmeret, IL); Jehuda Shemer (Kfar Saba, IL); Alex Solan (Hertzliya, IL)
Assignee: EMC IP HOLDING COMPANY LLC
G06F9/45558G06F11/203
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,619,264
App. No.
13/599,235
Filed
Aug 30, 2012
Granted
Apr 11, 2017
Kind
B1
Art Unit
2168
USPC
707/602
Abstract

A computer implemented method, system, and computer program product for recovering from a crash of a system being replicated, the method comprising determining the amount of recovery time due to the crash of each of a set of hypervisors; wherein each of the hypervisors runs one or more data replication elements selected from the group consisting of a splitter and a replication appliance; wherein each of the splitters and replication appliances replicates one or more volumes, creating an assignment of the one or more volumes to the set of replication appliances and creating an assignment of one or more replication appliances to a set of hypervisors to minimize the amount of recovery time.

Claims (48)

1. A computer implemented method for recovering from a crash of a system being replicated, the method comprising:

determining the amount of recovery time for a set of consistency groups due to a crash of each of a set of hypervisors; wherein the recovery time corresponds to amount of time to resynchronize data being replicated by the consistency group due to the crash of each of the set of hypervisors; wherein each of the hypervisors runs one or more data replication elements selected from a group consisting of a splitter and a replication appliance for the set of consistency group; wherein each of the splitters and replication appliances replicates one or more volumes of the set of consistency groups containing one or more replication sets; wherein the splitter intercepts IO written to one or more volumes and replicates a copy of the intercepted IO; wherein a volume running in a crashed hypervisor being replicated before the crash to a data replication appliance in the crashed hypervisor needs to be resynchronized; wherein a volume running in the crashed hypervisor being replicated to a data replication appliance in a different hypervisor does not need to be resynchronized;

creating an assignment of the one or more volumes to the replication appliances for the set of consistency groups and creating an assignment of one or more replication appliances to the set of hypervisors for the set of consistency groups to minimize an amount of recovery time by minimizing the amount of data needed to be resynchronized for each consistency group; wherein the assignment of the one or more volumes to the set of replication appliances denotes which replication appliance replicates which of the one or more volumes; wherein the assignment of one or more replication appliances to the set of hypervisors denotes which replication appliance runs on which hypervisor.

2. The computer implemented method of claim 1 further comprising:

moving one or more of the volumes to another replication appliance running on another hypervisor to minimize recovery time in the event of a crash of one of the hypervisors of the set of hypervisors; wherein moving the volume to a different replication appliance lowers the amount of recovery time for the hypervisor running the replication appliance.

3. The computer implemented method of claim 2 wherein the volumes are assigned to one or more consistency groups and wherein the moving comprises:

moving one or more consistency groups from a replication appliance running on a first hypervisor to a second replication appliance running on a second hypervisor to minimize recovery time in the event of a crash of one of the hypervisors of the set of hypervisors; wherein moving the consistency group to a different replication appliance lowers the amount of recovery time for the hypervisor running the replication appliance.

4. The computer implemented method of claim 1 wherein the creating comprises:

using an optimization technique selected from a group consisting of simulated annealing, genetic algorithms, genetic programming, and neural networks to create the assignment.

5. The computer implemented method of claim 4 wherein the determining the amount of recovery time comprises:

finding for each replication appliance, for each hypervisor of the set of hypervisors, the load on the replication appliance wherein the load include the amount of data being replicated by the replication appliance.

6. The computer implemented method of claim 4 wherein the determining the amount of recovery time comprises:

determining, for each hypervisor of the set of hypervisors, the amount of data replicated by the splitter and an appliance running on the hypervisor; wherein the amount of data replicated by a splitter corresponds to the amount of IO intercepted and split by the splitter.

7. The computer implemented method of claim 1 further comprising moving a virtual appliance from one hypervisor to another hypervisor using vmotion; wherein the virtual appliance is running in a virtual machine and the vmotion moves the virtual machine running the virtual appliance.

8. A computer program product for recovering from a crash of a system being replicated comprising:

a non-transitory computer readable medium encoded with computer executable program code for replication of data, the computer executable program code executed across one or more processors to perform:

determining the amount of recovery time for a set of consistency groups due to a crash of each of a set of hypervisors; wherein the recovery time corresponds to amount of time to resynchronize data being replicated by the consistency group due to the crash of each of the set of hypervisors; wherein each of the hypervisors runs one or more data replication elements selected from a group consisting of a splitter and a replication appliance for the set of consistency group; wherein each of the splitters and replication appliances replicates one or more volumes of the set of consistency groups containing one or more replication sets; wherein the splitter intercepts IO written to one or more volumes and replicates a copy of the intercepted IO; wherein a volume running in a crashed hypervisor being replicated before the crash to a data replication appliance in the crashed hypervisor needs to be resynchronized; wherein a volume running in the crashed hypervisor being replicated to a data replication appliance in a different hypervisor does not need to be resynchronized;

creating an assignment of the one or more volumes to the replication appliances for the set of consistency groups and creating an assignment of one or more replication appliances to the set of hypervisors for the set of consistency groups to minimize the amount of recovery time by minimizing the amount of data needed to be resynchronized for each consistency group; wherein the assignment of the one or more volumes to the set of replication appliances denotes which replication appliance replicates which of the one or more volumes; wherein the assignment of one or more replication appliances to the set of hypervisors denotes which replication appliance runs on which hypervisor.

9. The computer program product of claim 8 the computer executable program code further executed to perform:

moving one or more of the volumes to a different replication appliance running on a different hypervisor to minimize recovery time in the event of a crash of one of the hypervisors of the currently amended of hypervisors; wherein moving the volume to a different replication appliance lowers the amount of recovery time for the hypervisor running the replication appliance.

10. The computer program product of claim 9 wherein the volumes are assigned to one or more consistency groups and the computer executable program code further executed to perform:

moving one or more consistency groups from a replication appliance running on a first hypervisor to a second replication appliance running on a second hypervisor to minimize recovery time in the event of a crash of one of the hypervisors of the set of hypervisors wherein moving the consistency group to a different replication appliance lowers the amount of recovery time for the hypervisor running the replication appliance.

11. The computer program product of claim 8 wherein the computer executable program code further executed to perform:

using an optimization technique selected from a group consisting of simulated annealing, genetic algorithms, genetic programming, and neural networks to create the assignment.

12. The computer program product of claim 11 the computer program product code further executed to perform:

finding each replication appliance, for each hypervisor of the set of hypervisors, the load on the replication appliance wherein the load include the amount of data being replicated by the replication appliance.

13. The computer program product of claim 11 the computer program product code further executed to perform:

determining, for each hypervisor of the set of hypervisors, the amount of data replicated by the splitter and an appliance running on the hypervisor; wherein the amount of data replicated by a splitter corresponds to the amount of IO intercepted and split by the splitter.

14. The computer program product of claim 8 the computer program product code further executed to perform:

moving a virtual appliance from one hypervisor to another hypervisor using vmotion; wherein the virtual appliance is running in a virtual machine and the vmotion moves the virtual machine running the virtual appliance.

15. A system for recovering from a crash of a system being replicated, the system comprising:

a set of hypervisors;

one or more replication elements;

one or more processors;

one or more memories and

computer-executable program loaded in the one or more memory, wherein the computer-executable program is executed across the one or more processors to perform:

determining the amount of recovery time for a set of consistency groups due to a crash of each of the set of hypervisors; wherein the recovery time corresponds to amount of time to resynchronize data being replicated by the consistency group due to the crash of each of the set of hypervisors; wherein each of the hypervisors runs one or more data replication elements selected from a group consisting of a splitter and a replication appliance for the set of consistency group containing one or more replication sets; wherein each of the splitters and replication appliances replicates one or more volumes for the set of consistency group; wherein the splitter intercepts IO written to one or more volumes and replicates a copy of the intercepted IO; wherein a volume running in a crashed hypervisor being replicated before the crash to a data replication appliance in the crashed hypervisor needs to be resynchronized; wherein a volume running in the crashed hypervisor being replicated to a data replication appliance in a different hypervisor does not need to be resynchronized;

creating an assignment of the one or more volumes to the replication appliances for the set of consistency groups and creating an assignment of one or more replication appliances to the set of hypervisors for the set of consistency groups to minimize an amount of recovery time by minimizing the amount of data needed to be resynchronized for each consistency group; wherein the assignment of the one or more volumes to the set of replication appliances denotes which replication appliance replicates which of the one or more volumes; wherein the assignment of one or more replication appliances to the set of hypervisors denotes which replication appliance runs on which hypervisor.

16. The system of claim 15 the computer-executable program is further executed to perform:

moving one or more of the volumes to a different replication appliance running on a different hypervisor to minimize recovery time in the event of a crash of one of the hypervisors of the set of hypervisors; wherein moving the volume to a different replication appliance lowers the amount of recovery time for the hypervisor running the replication appliance.

17. The system of claim 16 wherein the volumes are assigned to one or more consistency groups and the computer-executable program is further executed to perform:

moving one or more consistency groups from a replication appliance running on a first hypervisor to a second replication appliance running on a second hypervisor to minimize recovery time in the event of a crash of one of the hypervisors of the set of hypervisors; wherein moving the consistency group to a different replication appliance lowers the amount of recovery time for the hypervisor running the replication appliance.

18. The system of claim 15 wherein the computer-executable program is further executed to perform:

using optimization technique selected from the group consisting of simulated annealing, genetic algorithms, genetic programming, and neural networks to create the assignment.

19. The system of claim 18 wherein the computer-executable program is further executed to perform:

finding each replication appliance, for each hypervisor of the set of hypervisors, the load on the replication appliance wherein the load include the amount of data being replicated by the replication appliance.

20. The system of claim 18 wherein the computer-executable program is further executed to perform:

determining, for each hypervisor of the set of hypervisors, the amount of data replicated by the splitter and an appliance running on the hypervisor; wherein the amount of data replicated by a splitter corresponds to the amount of IO intercepted and split by the splitter.

Assignments (11)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2013
From: EMC INTERNATIONAL COMPANY
To: EMC CORPORATION
Reel/Frame 030847/0412 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2012
From: NATANZON, ASSAF; COHEN, SAAR; SHEMER, JEHUDA; SOLAN, ALEX
To: EMC INTERNATIONAL COMPANY
Reel/Frame 028876/0052 →
Continuity (1)
Continuation In Part 13534042 · Jun 27, 2012