IP Library Granted Patent US 12,657,288
Granted Patent B2
US 12,657,288 · App. 17/550,516 · Granted Jun 16, 2026

Protection of call stack

Inventors: Naijie Li (Haidian Dist, CN); Jing Lu (Beijing, CN); Ming Ran Liu (Beijing, CN); Xiao Yan Tang (Beijing, CN); Yuan Zhai (Beijing, CN); Guang Han Sui (Beijing, CN)
Assignee: International Business Machines Corporation
G06F21/54G06F2221/033
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,657,288
App. No.
17/550,516
Granted
Jun 16, 2026
Kind
B2
Abstract

A first memory page in a memory of the computer is allocated as a first stack to buffer meta data for function calls in the program. A memory protection key for the first memory page is generated. A second memory page in the memory is allocated as a second stack to buffer user data for function calls in the program.

Claims (53)

1 . A method for executing a program in a computer, comprising:

allocating, by one or more processing units, a first memory page in a memory of the computer as a first stack to buffer function meta data for function calls of a function in the program;

generating, by the one or more processing units, a memory protection key for the first memory page; and

allocating, by the one or more processing units, a second memory page in the memory as a second stack to buffer function user data for the function calls of the function in the program, wherein the function is configured to call at least one callee function of the function and the allocating the first stack buffers meta data for the callee function and the allocating the second stack buffers user data for the function calls of the callee function.

2 . The method of claim 1 , further comprising:

in response to the function being invoked,

obtaining, by the one or more processing units, via a prologue of the function, the memory protection key; and

saving, by the one or more processing units, via the prologue of the function, meta data of the function to a stack frame in the first stack by using the memory protection key, the meta data comprising context information of a caller invoking the function.

3 . The method of claim 2 , further comprising:

storing, by the one or more processing units, user data of the function to a stack frame in the second stack, the user data being generated during execution of the function.

4 . The method of claim 3 , further comprising:

in response to the execution of the function being completed:

obtaining, by the one or more processing units, via an epilogue of the function, the memory protection key;

retrieving, by the one or more processing units, via the epilogue, the meta data of the function from a stack frame in the first stack by using the memory protection key; and

restoring, by the one or more processing units, via the epilogue, the context information for the caller with the meta data.

5 . The method of claim 4 , further comprising:

releasing, by the one or more processing units, the stack frame in the first stack and a second stack frame in the second stack from the memory upon exiting of the function.

6 . The method of claim 1 , wherein the first stack and the second stack are First-In Last Out stacks.

7 . The method of claim 1 , wherein the memory protection key is used for a second function of the program.

8 . A system, comprising:

one or more processors; and

one or more computer-readable storage media storing program instructions which, when executed by the one or more processors, are configured to cause the one or more processors to perform the following operations:

allocating a first memory page in a memory of a computer as a first stack to buffer function meta data for function calls of a function in the program;

generating a memory protection key for the first memory page; and

allocating a second memory page in the memory as a second stack to buffer function user data for the function calls of the function in the program, wherein the function is configured to call at least one callee function of the function and the allocating the first stack buffers meta data for the callee function and the allocating the second stack buffers user data for the function calls of the callee function.

9 . The system of claim 8 , the program instructions being further configured to perform the following operations in response to the function being invoked:

obtaining, via a prologue of the function, the memory protection key; and

saving, via the prologue of the function, meta data of the function to a stack frame in the first stack by using the memory protection key, the meta data comprising context information of a caller invoking the function.

10 . The system of claim 9 , the program instructions being further configured to perform the following operation:

storing, by one or more processing units, user data of the function to a stack frame in the second stack, the user data being generated during execution of the function.

11 . The system of claim 10 , the program instructions being further configured to perform the following operations in response to the execution of the function being completed:

obtaining, via an epilogue of the function, the memory protection key;

retrieving, via the epilogue, the meta data of the function from a stack frame in the first stack by using the memory protection key; and

restoring, via the epilogue, the context information for the caller with the meta data.

12 . The system of claim 10 , the program instructions being further configured to perform the following operation:

releasing, via an epilogue of the function, the stack frame in the first stack and a second stack frame in the second stack from the memory upon exiting of the function.

13 . The system of claim 8 , wherein the first stack and second stack are First-In Last Out stacks.

14 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by processing means, so as to cause the processing means to perform the following operations:

allocating a first memory page in a memory of the computer as a first stack to buffer function meta data for a first execution of function calls of a function in the program;

generating a memory protection key for the first memory page; and

allocating a second memory page in the memory as a second stack to buffer function user data for the first execution of the function calls of the function in the program.

15 . The computer program product of claim 14 , the program instructions being further configured to cause the processing means perform the following operations in response to the function being invoked:

obtaining, via a prologue of the function, the memory protection key; and

saving, via the prologue of the function, meta data of the function to a stack frame in the first stack by using the memory protection key, the meta data comprising context information of a caller invoking the function.

16 . The computer program product of claim 15 , the program instructions being further configured to cause the processing means perform the following operation:

storing, by one or more processing units, user data of the function to a stack frame in the second stack, the user data being generated during execution of the function.

17 . The computer program product of claim 16 , the program instructions being further configured to cause the processing means perform the following operations in response to the execution of the function being completed:

obtaining, via an epilogue of the function, the memory protection key;

retrieving, via the epilogue, the meta data of the function from a stack frame in the first stack by using the memory protection key; and

restoring, via the epilogue, the context information for the caller with the meta data.

18 . The computer program product of claim 16 , further configured to cause the processing means perform the following operation:

releasing, via an epilogue of the function, the stack frame in the first stack and a second stack frame in the second stack from the memory upon exiting of the function.

19 . The computer program product of claim 14 , wherein the first stack and second stack are First-In Last Out stacks.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2021
From: LI, NAIJIE; LU, JING; LIU, MING RAN; TANG, XIAO YAN; ZHAI, YUAN; SUI, GUANG HAN
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 058387/0488 →
Continuity (1)
Related Publication 20230185903A1 · Jun 15, 2023
References Cited (17)
US 7581089B1 · White · 2009 [cited by applicant]
US 7603704B2 · Bruening et al. · 2009 [cited by applicant]
US 9037872B2 · Kaplan · 2015 [cited by applicant]
US 9251373B2 · AlHarbi et al. · 2016 [cited by applicant]
US 9495237B1 · Gschwind et al. · 2016 [cited by applicant]
US 11372969B1 · Sundahl · 2022 [cited by examiner]
US 20120054582A1 · Byom · 2012 [cited by examiner]
US 20120278635A1 · Hars · 2012 [cited by examiner]
US 20160098556A1 · Alharbi et al. · 2016 [cited by applicant]
US 20200042180A1 · Byun · 2020 [cited by examiner]
US 20210240638A1 · Deutsch · 2021 [cited by examiner]
US 20220100418A1 · Benisty · 2022 [cited by examiner]
US 20240143206A1 · Arbel · 2024 [cited by examiner]
Zhao et al., “MPTEE: Bringing Flexible and Efficient Memory Protection to Intel SGX,” EuroSys '20, Apr. 27-30, 2020, Heraklion, Greece, 15 pages, https://doi.org/10.1145/3342195.3387536. [cited by applicant]
Jin et al., “BoundShield: Comprehensive Mitigation for Memory Disclosure Attacks via Secret Region Isolation,” IEEE Access, May 14, 2018, 13 pages, doi. 10.1109/ACCESS.2018.2835838. [cited by applicant]
Younan et al., “Extended protection against stack smashing attacks without performance loss,” Research Gate, Dec. 2006, 11 pages, https://www.researchgate.net/publication/221046682. [cited by applicant]
Miller Matt, “Trends, challenges and strategic shifts in the software vulnerability mitigation landscape”, BlueHat IL, Feb. 7, 2019, 32 pages. [cited by applicant]