IP Library › Granted Patent US 9,104,645
Granted Patent B2
US 9,104,645 · App. 13/560,482 · Granted Aug 11, 2015

System and method of replicating virtual machines for live migration between data centers

Inventors: William P. Dawkins (Austin, TX); Gaurav Chawla (Austin, TX)
Assignee: Dell Products, LP
G06F11/203G06F9/45558G06F11/1484G06F11/2069G06F11/2074G06F11/2082G06F11/2038G06F2009/4557
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,104,645
App. No.
13/560,482
Filed
Jul 27, 2012
Granted
Aug 11, 2015
Kind
B2
Art Unit
2199
USPC
718/1
Abstract

A method includes providing a storage volume at a processing site, directing a storage controller of the processing site to asynchronously mirror the storage volume to another storage volume of another processing site that is remote from the first processing site, determining to migrate a virtual machine of the first processing site that is associated with the first storage volume to the second processing site, synchronizing the first and second storage volumes in response to the migration, live migrating the virtual machine to the second processing site and a second virtual machine of the second processing site with the second storage volume, and directing a second storage controller of the second processing site to asynchronously mirror the second storage volume to the first storage volume.

Claims (50)

1. A method comprising:

providing, by a storage controller, a first storage volume at a first processing site;

providing a first virtual machine at the first processing site, wherein the first virtual machine is associated with the first storage volume;

in response to providing the first virtual machine, directing the storage controller to mirror the first storage volume to a second storage volume of a second processing site, wherein the storage controller mirrors the first storage volume to the second storage volume asynchronously, and wherein the second processing site is remote from the first processing site;

after directing the storage controller to minor the first storage volume to the second storage volume, determining to migrate the first virtual machine to the second processing site;

synchronizing the second storage volume with the first storage volume in response to determining to migrate the first virtual machine, wherein the synchronizing comprises switching from asynchronous replication to direct synchronous replication;

while performing the direct synchronous replication of the second storage volume with the first storage volume, operating the first virtual machine such that a storage operation of the first virtual machine is directed to the first storage volume;

receiving an indication that the first storage volume and the second storage volume are synchronized;

in response to receiving the indication, by the storage controller, live migrating the first virtual machine, by a virtualization controller, to the second processing site, wherein the live migration operates to associate a second virtual machine of the second processing site with the second storage volume; and

directing the storage controller to mirror the second storage volume to the first storage volume, wherein the storage controller mirrors the second storage volume to the first storage volume asynchronously, and wherein the directing comprises switching from the direct synchronous replication to asynchronous replication.

2. The method of claim 1 , wherein the asynchronous mirroring of the first storage volume to the second storage volume is performed in response to a first checkpoint.

3. The method of claim 2 , wherein synchronizing the second storage volume with the first storage volume comprises sending data written to the first storage volume after a prior checkpoint to the second storage volume.

4. The method of claim 2 , wherein the checkpoint is determined in response to a predetermined time interval.

5. The method of claim 1 , wherein the first storage volume is assigned a first logical unit number associated with the first virtual machine.

6. The method of claim 5 , wherein the second storage volume is assigned a second logical unit number and is associated with the second virtual machine.

7. The method of claim 1 , wherein determining to migrate the first virtual machine to the second processing site is in response to load balancing between the first processing site and the second processing site.

8. A storage controller comprising:

a memory; and

a processor operable to:

provide a first storage volume at a first processing site;

provide a first virtual machine at the first processing site, wherein the first virtual machine is associated with the first storage volume;

in response to providing the first virtual machine, minor the first storage volume to a second storage volume of a second processing site, wherein the storage controller mirrors the first storage volume to the second storage volume asynchronously, wherein the processor establishes a checkpoint interval for the asynchronous mirroring of the first storage volume to the second storage volume, and wherein the second processing site is remote from the first processing site;

after directing the storage controller to minor the first storage volume to the second storage volume, determine to migrate the first virtual machine to the second processing site;

in response to determining to migrate the first virtual machine to the second processing site, send data that has been changed on the first storage volume subsequent to a last previous checkpoint interval to the second storage volume;

synchronize the second storage volume with the first storage volume in response to determining to migrate a first virtual machine to the second processing site, wherein the synchronizing comprises switching from asynchronous replication to direct synchronous replication;

while performing the direct synchronous replication of the second storage volume with the first storage volume, operate the first virtual machine such that a storage operation of the first virtual machine is directed to the first storage volume;

receive an indication that the first storage volume and the second storage volume are synchronized;

in response to receiving the indication, live migrate the first virtual machine to the second processing site, wherein the live migration operates to associate a second virtual machine of the second processing site with the second storage volume; and

mirror the second storage volume to the first storage volume in response to the live migration of the first virtual machine to the second processing site wherein the second storage volume is mirrored to the first storage volume asynchronously, and wherein the mirroring comprises switching from the direct synchronous replication to the asynchronous replication.

9. The storage controller of claim 8 , wherein the asynchronous mirroring of the first storage volume to the second storage volume is performed in response to a first checkpoint.

10. The storage controller of claim 9 , wherein synchronizing the second storage volume with the first storage volume comprises sending data written to the first storage volume after a prior checkpoint to the second storage volume.

11. The storage controller of claim 9 , wherein the checkpoint is determined in response to a predetermined time interval.

12. The storage controller of claim 8 , wherein the processor is further operable to assign a first logical unit number associated with the first virtual machine to the first storage volume.

13. The storage controller of claim 12 , wherein the processor is further operable to assign a second logical unit number to the second storage volume.

14. The storage controller of claim 8 , wherein determining to migrate the first virtual machine to the second processing site is in response to load balancing between the first processing site and the second processing site.

15. A non-transitory computer-readable medium including code for performing a method, the method comprising:

providing, by a storage controller, a first storage volume at a first processing site;

providing a first virtual machine at the first processing site, wherein the first virtual machine is associated with the first storage volume;

in response to providing the first virtual machine, directing the storage controller of the first processing site to mirror the first storage volume to a second storage volume of a second processing site, wherein the storage controller mirrors the first storage volume to the second storage volume asynchronously, and wherein the second processing site is remote from the first processing site;

after directing the storage controller to minor the first storage volume to the second storage volume, determining to migrate the first virtual machine to the second processing site;

synchronizing the second storage volume with the first storage volume in response to determining to migrate the first virtual machine, wherein the synchronizing comprises switching from asynchronous replication to direct synchronous replication;

while performing the direct synchronous replication of the second storage volume with the first storage volume, operating the first virtual machine such that a storage operation of the first virtual machine is directed to the first storage volume and to the second storage volume;

receiving an indication that the first storage volume and the second storage volume are synchronized;

in response to receiving the indication, live migrating the first virtual machine to the second processing site, wherein the live migration operates to associate a second virtual machine of the second processing site with the second storage volume; and

directing the storage controller to mirror the second storage volume to the first storage volume, wherein the storage controller mirrors the second storage volume to the first storage volume asynchronously, and wherein the mirroring comprises switching from the direct synchronous replication to the asynchronous replication.

16. The computer-readable medium of claim 15 , wherein the asynchronous mirroring of the first storage volume to the second storage volume is performed in response to a first checkpoint.

17. The computer-readable medium of claim 16 , wherein synchronizing the second storage volume with the first storage volume comprises sending data written to the first storage volume after a prior checkpoint to the second storage volume.

18. The computer-readable medium of claim 15 , wherein the first storage volume is assigned a first logical unit number associated with the first virtual machine.

19. The computer-readable medium of claim 18 , wherein the second storage volume is assigned a second logical unit number and is associated with the second virtual machine.

20. The computer-readable medium of claim 15 , wherein determining to migrate the first virtual machine to the second processing site is in response to load balancing between the first processing site and the second processing site.

Assignments (15)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 14, 2016
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELL MARKETING L.P.; ASAP SOFTWARE EXPRESS, INC.; APPASSURE SOFTWARE, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL PRODUCTS L.P.; DELL USA L.P.; DELL SOFTWARE INC.; FORCE10 NETWORKS, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040040/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 14, 2016
From: BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: DELL MARKETING L.P.; ASAP SOFTWARE EXPRESS, INC.; APPASSURE SOFTWARE, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL PRODUCTS L.P.; DELL USA L.P.; DELL SOFTWARE INC.; FORCE10 NETWORKS, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040065/0618 →
RELEASE OF SECURITY INTEREST Recorded Sep 13, 2016
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: DELL MARKETING L.P.; ASAP SOFTWARE EXPRESS, INC.; APPASSURE SOFTWARE, INC.; COMPELLANT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL PRODUCTS L.P.; DELL USA L.P.; DELL SOFTWARE INC.; FORCE10 NETWORKS, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040065/0216 →
PATENT SECURITY AGREEMENT (ABL) Recorded Jan 2, 2014
From: DELL INC.; APPASSURE SOFTWARE, INC.; ASAP SOFTWARE EXPRESS, INC.; BOOMI, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL USA L.P.; FORCE10 NETWORKS, INC.; GALE TECHNOLOGIES, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 031898/0001 →
PATENT SECURITY AGREEMENT (TERM LOAN) Recorded Jan 2, 2014
From: DELL INC.; APPASSURE SOFTWARE, INC.; ASAP SOFTWARE EXPRESS, INC.; BOOMI, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL USA L.P.; FORCE10 NETWORKS, INC.; GALE TECHNOLOGIES, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 031899/0261 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Jan 2, 2014
From: APPASSURE SOFTWARE, INC.; ASAP SOFTWARE EXPRESS, INC.; BOOMI, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL USA L.P.; FORCE10 NETWORKS, INC.; GALE TECHNOLOGIES, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS FIRST LIEN COLLATERAL AGENT
Reel/Frame 031897/0348 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 1, 2012
From: DAWKINS, WILLIAM P.; CHAWLA, GAURAV
To: DELL PRODUCTS, LP
Reel/Frame 028695/0562 →
Continuity (1)
Related Publication 20140033201A1 · Jan 30, 2014