IP Library Granted Patent US 10,216,598
Granted Patent B2
US 10,216,598 · App. 15/646,769 · Granted Feb 26, 2019

Method for dirty-page tracking and full memory mirroring redundancy in a fault-tolerant server

Inventors: Steven Michael Haid (Bolton, MA); John Rogers MacLeod (Cambridge, MA)
Assignee: Stratus Technologies Bermuda LTD.
G06F11/2097G06F11/1438G06F11/1451G06F11/1482G06F11/203G06F2212/152
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,216,598
App. No.
15/646,769
Granted
Feb 26, 2019
Kind
B2
Abstract

A method of transferring memory from an active to a standby memory in an FT Server system. The method includes the steps of: reserving a portion of memory using BIOS; loading and initializing an FT Kernel Mode Driver; loading and initializing an FT Virtual Machine Manager (FTVMM) including the Second Level Address Translation table SLAT into the reserved memory. In another embodiment, the method includes tracking memory accesses using the FTVMM's SLAT in Reserved Memory and tracking “L 2 ” Guest memory accesses by tracking the current Guest's SLAT and intercepting the Hypervisor's writes to the SLAT. In yet another embodiment, the method includes entering Brownout by collecting the D-Bits; invalidating the processor's cached SLAT translation entries, and copying the dirtied pages from the active memory to memory in the second Subsystem. In one embodiment, the method includes entering Blackout and moving the final dirty pages from active to the mirror memory.

Claims (52)

1. A method of transferring memory from an active to a standby memory in an FT (Fault Tolerant) Server system comprising the steps of:

Reserving a portion of memory using BIOS of the FT Server system;

Loading and initializing an FT Kernel Mode Driver into memory;

Loading and initializing an FT virtual machine Manager (FTVMM) including the Second Level Address Table (SLAT) into the reserved portion of memory and synchronizing all processors in the FTVMM;

Tracking the OS (Operating System), driver, software and Hypervisor memory accesses using the FTVMM's SLAT in Reserved Memory;

Tracking Guest VM (Virtual Machine) memory accesses by tracking all pages of the SLAT associated with the guest and intercepting the Hypervisor writes to memory pages that constitute the SLAT;

Entering Brownout—level 0 , by executing a full memory copy while keeping track of the dirty bits (D-Bits) in the SLAT to track memory writes by all software in the FT Server;

Clearing all of the D-Bits in the FTVMM SLAT and each Guest's current SLAT;

Entering Brownout of phases 1 - 4 tracking all D-Bits by:

Collecting the D-Bits;

Invalidating all processors cached translations for the FTVMM SLAT and each current Guest's SLAT;

Copying the data of the modified memory pages from the active memory to the second Subsystem memory;

Entering Blackout by pausing all the processors in the FTVMM, except Processor # 0 , and disabling interrupts in the FT Driver for processor # 0 ;

Copying the collected data from active to the mirror memory;

Collecting the final set of recently dirtied pages including stack and volatile pages;

Transferring the set of final pages to the mirror memory using an FPGA (Field Programmable Gate Array);

Using the FT Server FPGA and Firmware SMM (System Management Module) to enter the state of Mirrored Execution;

Completing the Blackout portion of the operation by terminating and unloading the FTVMM;

Returning control to the FT Kernel Mode Driver; and

Resuming normal FT System operation.

2. The method of claim 1 wherein FTVMM code remains in Reserved Memory following the resumption of normal FT System operation.

3. The method of claim 1 wherein the reserved memory is a memory region from ¼ GByte to 2.25 GByte of contiguous physical memory, located up to and including the maximum existing physical memory address in the FT Server.

4. The method of claim 1 wherein moving the data from the active memory uses an FPGA.

5. The method of claim 1 wherein the return of control to the FT Kernel Mode Driver is by the FT Kernel Mode Driver causing a VMEXIT event to occur.

6. The method of claim 1 wherein the FT Kernel Mode Driver returns to the condition of normal Guest VM operation on each processor by performing a VMPTRLD (Virtual Machine Pointer Load) instruction with the current guest VMCS (VM Control Structure) as an operand to make the current guest VMCS current and active.

7. The method of claim 1 where the collecting of D-bits is performed either periodically from the SLAT or continuously using Page Modification Logging (PML).

8. A method of transferring memory from an active to a standby memory in an FT (Fault Tolerant) Server system comprising the steps of:

Reserving a portion of memory using BIOS of the FT Server system at boot time;

Loading and initializing an FT Kernel Mode Driver;

Loading and initializing an FT virtual machine Manager (FTVMM) including the SLAT into the reserved portion of memory and synchronizing all processors in the FTVMM;

Tracking memory accesses using the SLAT in Reserved Memory;

Entering Brownout—level 0 , by executing a full memory copy while keeping track of the dirty bits (D-Bits) in the SLAT;

Clearing all of the D-Bits in the SLAT;

Entering Brownout of phases 1 - 4 tracking all D-Bits by:

Collecting the D-Bits;

Invalidating all processors cached translations for SLAT;

Copying the data of the modified memory pages from the active memory to the second Subsystem memory;

Entering Blackout by pausing all the processors in the FTVMM, except Processor # 0 , and disabling interrupts in the FT Driver for processor # 0 ;

Copying the collected data from active to the mirror memory;

Collecting the final set of recently dirtied pages;

Transferring the set of final pages to the mirror memory using an FPGA (Field Programmable Gate Array);

Using the FT Server FPGA and Firmware SMM to enter the state of Mirrored Execution;

Unloading the FTVMM and exiting into FT Kernel Mode Driver to cease operation of the FTVMM.

9. The method of claim 8 wherein synchronizing of all processors in the FTVMM causes FTVMM code to execute as hypervisor on all processors.

10. The method of claim 8 wherein the FTVMM tracks memory accesses using the SLAT in Reserved Memory.

11. The method of claim 8 wherein the full memory copy of system memory to the standby memory is performed by the FT Driver.

12. The method of claim 11 , wherein the copying the data of the modified memory pages from the active memory to the second Subsystem memory begins with the FTVMM obtaining a copy of all pages dirtied during the full memory copy by the FT Driver.

13. The method of claim 12 , wherein the FTVMM provides dirty page bit map data to the FT Driver.

14. The method of claim 13 , wherein the providing of the dirty page bit map to the FT Driver occurs for any pages dirtied during the previous copying of dirty page data by the FT Driver.

15. The method of claim 8 , wherein the SMM enters the state of Mirrored Execution by restoring Processor State from memory.

16. The method of claim 15 , wherein the SMM returns to the FT Driver after Processor State is restored, completing entrance into Mirrored Execution.

17. The method of claim 8 where the collecting of D-bits is performed either periodically from the SLAT or continuously using PML.

Assignments (5)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (057254/0557) Recorded Aug 29, 2022
From: CERBERUS BUSINESS FINANCE AGENCY, LLC
To: STRATUS TECHNOLOGIES IRELAND LIMITED; STRATUS TECHNOLOGIES BERMUDA LTD.
Reel/Frame 061354/0599 →
GRANT OF SECURITY INTEREST IN PATENT RIGHTS Recorded Jun 9, 2021
From: STRATUS TECHNOLOGIES IRELAND LIMITED; STRATUS TECHNOLOGIES BERMUDA LTD.
To: CERBERUS BUSINESS FINANCE AGENCY, LLC, AS COLLATERAL AGENT
Reel/Frame 057254/0557 →
SECURITY INTEREST Recorded Apr 3, 2020
From: STRATUS TECHNOLOGIES IRELAND LIMITED
To: TRUST BANK (AS SUCCESSOR BY MERGER TO SUNTRUST BANK)
Reel/Frame 052316/0371 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2020
From: STRATUS TECHNOLOGIES BERMUDA LTD.
To: STRATUS TECHNOLOGIES IRELAND LTD.
Reel/Frame 052210/0411 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2017
From: HAID, STEVEN MICHAEL; MACLEOD, JOHN ROGERS
To: STRATUS TECHNOLOGIES BERMUDA LTD.
Reel/Frame 043297/0929 →
Continuity (1)
Related Publication 20190018746A1 · Jan 17, 2019
Cited By (7)
US 12,326,811 US 12,405,868 US 12,463,949 US 12,475,006 US 12,625,783 US 12,632,280 US 12,711,030