IP Library Granted Patent US 11,586,727
Granted Patent B2
US 11,586,727 · App. 17/216,188 · Granted Feb 21, 2023

Systems and methods for preventing kernel stalling attacks

Inventor: Michael Tsirkin (Yokneam Illit, IL)
Assignee: Red Hat, Inc.
G06F21/53G06F2221/033
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,586,727
App. No.
17/216,188
Granted
Feb 21, 2023
Kind
B2
Abstract

Systems and methods for preventing kernel stalling attacks. An example method may comprise receiving, by a kernel, an address range associated with a data store of an application program; mapping, by the kernel, a portion of random access memory (RAM) to the address range; disabling page fault handling with respect to addresses falling within the address range; and responsive to receiving, from the application program, a memory access request specifying an address outside of the address range, returning a memory access error to the application program.

Claims (54)

1. A method comprising:

receiving, by a kernel, an address range associated with a data store of an application program;

mapping, by the kernel, a portion of random access memory (RAM) to the address range;

disabling page fault handling with respect to addresses falling within the address range;

maintaining a first memory protection key and a second memory protection key, wherein the first memory protection key enables accessing user space memory and the second memory protection key is limited to the address range in the user space memory;

responsive to receiving, from the application program, a memory access request, switching from using the first protection key to the second protection key; and

responsive to determining that the memory access request specifies an address outside of the address range, returning a memory access error to the application program.

2. The method of claim 1 , further comprising:

responsive to receiving, from the application program, a memory access request specifying an address within the address range, processing the memory request.

3. The method of claim 1 , further comprising:

responsive to returning the error to the application program, enabling page fault handling with respect to the address.

4. The method of claim 1 , further comprising:

pinning one or more memory pages mapped to addresses falling within the address range.

5. The method of claim 1 , further comprising:

responsive to processing the memory access request, paging out data associated with the memory access request.

6. The method of claim 1 , further comprising:

maintaining a first page table and a second page table, wherein the first page table comprises a set of records, wherein each record of the set of records maps a virtual address in the address range to an address in physical memory,

responsive to receiving, from the application program, the memory access request, switching from the first page table to the second page table.

7. A system, comprising:

a memory;

a processing device operatively coupled to the memory, the processing device configured to:

receive an address range associated with an application program;

map a portion of random access memory (RAM) to the address range;

disable page fault handling with respect to addresses falling within the address range;

maintain a first memory protection key and a second memory protection key, wherein the first memory protection key enables accessing user space memory and the second memory protection key is limited to the address range in the user space memory;

responsive to receiving, from the application program, a memory access request, switch from using the first protection key to the second protection key; and

responsive to determining that the memory access request specifies an address outside of the address range, return a memory access error to the application program.

8. The system of claim 7 , further comprising the processing device configured to:

responsive to receiving, from the application program, a memory access request specifying an address within the address range, process the memory request.

9. The system of claim 7 , further comprising the processing device configured to:

responsive to returning the error to the application program, enable page fault handling with respect to the address falling within the address range.

10. The system of claim 7 , further comprising the processing device configured to:

pin one or more memory pages mapped to addresses falling within the address range.

11. The system of claim 7 , further comprising the processing device configured to:

responsive to processing the memory access request, page out data associated with the memory access request.

12. The system of claim 7 , further comprising the processing device configured to:

maintain a first page table and a second page table, wherein the first page table comprises a set of records, wherein each record of the set of records maps a virtual address in the address range to an address in physical memory,

responsive to receiving, from the application program, the memory access request, switch from the first page table to the second page table.

13. A non-transitory machine-readable storage medium storing instructions that cause a processing device to:

receive an address range associated with an application program;

map a portion of random access memory (RAM) to the address range;

disable page fault handling with respect to addresses falling within the address range;

maintain a first memory protection key and a second memory protection key, wherein the first memory protection key enables accessing user space memory and the second memory protection key is limited to the address range in the user space memory;

responsive to receiving, from the application program, a memory access request, switch from using the first protection key to the second protection key; and

responsive to determining that the memory access request specifies an address outside of the address range, return a memory access error to the application program.

14. The non-transitory machine-readable storage medium of claim 13 , further comprising the processing device configured to:

responsive to receiving, from the application program, a memory access request specifying an address within the address range, processing the memory request.

15. The non-transitory machine-readable storage medium of claim 13 , further comprising the processing device configured to:

responsive to returning the error to the application program, enabling page fault handling with respect to the address falling within the address range.

16. The non-transitory machine-readable storage medium of claim 13 , further comprising the processing device configured to:

pin one or more memory pages mapped to addresses falling within the address range.

17. The non-transitory machine-readable storage medium of claim 13 , further comprising the processing device configured to:

maintain a first page table and a second page table, wherein the first page table comprises a set of records, wherein each record of the set of records maps a virtual address in the address range to an address in physical memory,

responsive to receiving, from the application program, the memory access request, switch from the first page table to the second page table.

Assignments (2)
CHANGE OF NAME Recorded Mar 3, 2026
From: RED HAT, INC.
To: RED HAT, LLC
Reel/Frame 074913/0759 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2021
From: TSIRKIN, MICHAEL
To: RED HAT, INC.
Reel/Frame 055757/0972 →
Continuity (1)
Related Publication 20220309150A1 · Sep 29, 2022
Cited By (2)
US 12,430,259 US 12,682,069