IP Library Granted Patent US 12,253,956
Granted Patent B2
US 12,253,956 · App. 18/053,103 · Granted Mar 18, 2025

Hybrid scheme for performing translation lookaside buffer (TLB) shootdowns

Inventors: Andrei Warkentin (South Elgin, IL); Jared McNeill (Quispamsis, CA); Grant Foudree (Mountain View, CA); Anil Veliyankaramadam (San Ramon, CA)
Assignee: VMWare LLC
G06F12/1045G06F2212/68
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,253,956
App. No.
18/053,103
Granted
Mar 18, 2025
Kind
B2
Abstract

A hybrid scheme is provided for performing translation lookaside buffer (TLB) shootdowns in a computer system whose processing cores support both inter-processor interrupt (IPI) and broadcast TLB invalidate (TLBI) shootdown mechanisms. In one set of embodiments, this hybrid scheme dynamically determines, for each instance where a TLB shootdown is needed, whether to use the IPI mechanism or the broadcast TLBI mechanism to optimize shootdown performance (or otherwise make the TLB shootdown operation functional/practical).

Claims (56)

1. A method comprising:

determining, by an operating system (OS) kernel process running on a first processing core of a computer system, that a translation lookaside buffer (TLB) shootdown should be performed with respect to one or more second processing cores of the computer system;

flushing, by the OS kernel process, one or more page table entries (PTEs) from a TLB of the first processing core;

determining, by the OS kernel process, whether the TLB shootdown should be performed using an inter-processor interrupt (IPI) mechanism or a broadcast TLB invalidate (TLBI) mechanism;

sending, by the OS kernel process, TLB flush information to a shared system memory, the TLB flush information comprising a virtual address; and

performing, by the OS kernel process, the TLB shootdown using the determined mechanism;

wherein the one or more second processing cores read the TLB flush information from system memory and flush the corresponding stale PTEs.

2. The method of claim 1 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking a variable indicating whether the broadcast TLBI mechanism is broken on the computer system, and

wherein the TLB shootdown is performed using the IPI mechanism if the variable indicates that the broadcast TLBI mechanism is broken.

3. The method of claim 2 wherein the variable is set by an OS kernel of the computer system by:

retrieving a central processing unit (CPU) or platform identifier associated with the first and second processing cores; and

determining whether the CPU or platform identifier is mapped to an erratum indicating that the broadcast TLBI mechanism is broken.

4. The method of claim 1 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking whether an instruction barrier is needed at the one or more second processing cores, and

wherein the TLB shootdown is performed using the IPI mechanism if the instruction barrier is needed.

5. The method of claim 1 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking whether a total count of the one or more second processing cores exceeds a threshold, and

wherein the TLB shootdown is performed using the broadcast TLBI mechanism if the total count exceeds the threshold.

6. The method of claim 5 wherein the threshold is based on empirical evidence indicating performance tradeoffs when using the IPI mechanism and the broadcast TLBI mechanism on the computer system for different numbers of target processing cores.

7. The method of claim 1 wherein the computer system is a 64-bit ARM-based system.

8. A computer system comprising:

a plurality of processing cores, each including a translation lookaside buffer (TLB); and

a non-transitory computer readable medium having stored thereon program code for an operating system (OS) kernel process that, when executed by a first processing core in the plurality of processing cores, causes the first processing core to:

determine that a TLB shootdown should be performed with respect to one or more second processing cores in the plurality of processing cores;

flush, by the OS kernel process, one or more page table entries (PTEs) from a TLB of the first processing core;

determine whether the TLB shootdown should be performed using an inter-processor interrupt (IPI) mechanism or a broadcast TLB invalidate (TLBI) mechanism;

send, by the OS kernel process, TLB flush information to a shared system memory, the TLB flush information comprising a virtual address; and

perform the TLB shootdown using the determined mechanism;

wherein the one or more second processing cores read the TLB flush information from system memory and flush the corresponding stale PTEs.

9. The computer system of claim 8 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking a variable indicating whether the broadcast TLBI mechanism is broken on the computer system, and

wherein the TLB shootdown is performed using the IPI mechanism if the variable indicates that the broadcast TLBI mechanism is broken.

10. The computer system of claim 9 wherein the variable is set by an OS kernel of the computer system by:

retrieving a central processing unit (CPU) or platform identifier associated with the first and second processing cores; and

determining whether the CPU or platform identifier is mapped to an erratum indicating that the broadcast TLBI mechanism is broken.

11. The computer system of claim 8 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking whether an instruction barrier is needed at the one or more second processing cores, and

wherein the TLB shootdown is performed using the IPI mechanism if the instruction barrier is needed.

12. The computer system of claim 8 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking whether a total count of the one or more second processing cores exceeds a threshold, and

wherein the TLB shootdown is performed using the broadcast TLBI mechanism if the total count exceeds the threshold.

13. The computer system of claim 12 wherein the threshold is based on empirical evidence indicating performance tradeoffs when using the IPI mechanism and the broadcast TLBI mechanism on the computer system for different numbers of target processing cores.

14. The computer system of claim 8 wherein the plurality of processing cores implement a 64-bit ARM architecture.

15. A non-transitory computer readable storage medium having stored thereon instructions executable by an operating system (OS) kernel process running on a first processing core of a computer system, the instructions embodying a method comprising:

determining that a translation lookaside buffer (TLB) shootdown should be performed with respect to one or more second processing cores of the computer system;

flushing, by the OS kernel process, one or more page table entries (PTEs) from a TLB of the first processing core;

determining whether the TLB shootdown should be performed using an inter-processor interrupt (IPI) mechanism or a broadcast TLB invalidate (TLBI) mechanism;

sending, by the OS kernel process, TLB flush information to a shared system memory, the TLB flush information comprising a virtual address; and

performing the TLB shootdown using the determined mechanism;

wherein the one or more second processing cores read the TLB flush information from system memory and flush the corresponding stale PTEs.

16. The non-transitory computer readable storage medium of claim 15 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking a variable indicating whether the broadcast TLBI mechanism is broken on the computer system, and

wherein the TLB shootdown is performed using the IPI mechanism if the variable indicates that the broadcast TLBI mechanism is broken.

17. The non-transitory computer readable storage medium of claim 16 wherein the variable is set by an OS kernel of the computer system by:

retrieving a central processing unit (CPU) or platform identifier associated with the first and second processing cores; and

determining whether the CPU or platform identifier is mapped to an erratum indicating that the broadcast TLBI mechanism is broken.

18. The non-transitory computer readable storage medium of claim 15 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking whether an instruction barrier is needed at the one or more second processing cores, and

wherein the TLB shootdown is performed using the IPI mechanism if the instruction barrier is needed.

19. The non-transitory computer readable storage medium of claim 15 wherein determining whether the TLB shootdown should be performed using the IPI mechanism or the broadcast TLBI mechanism comprises checking whether a total count of the one or more second processing cores exceeds a threshold, and

wherein the TLB shootdown is performed using the broadcast TLBI mechanism if the total count exceeds the threshold.

20. The non-transitory computer readable storage medium of claim 19 wherein the threshold is based on empirical evidence indicating performance tradeoffs when using the IPI mechanism and the broadcast TLBI mechanism on the computer system for different numbers of target processing cores.

21. The non-transitory computer readable storage medium of claim 15 wherein the computer system is a 64-bit ARM-based system.

Assignments (2)
CHANGE OF NAME Recorded Feb 27, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 066692/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2022
From: WARKENTIN, ANDREI; MCNEILL, JARED; FOUDREE, GRANT; VELIYANKARAMADAM, ANIL
To: VMWARE INC.
Reel/Frame 061675/0503 →