Systems, methods, and apparatus for detecting control flow attacks

1. A data processing system with for detecting control flow attacks, the data processing system comprising:

a processor;

a performance monitoring unit (PMU) in the processor;

a heuristic event counter in the PMU;

a last branch record (LBR) buffer in the processor;

transaction hardware in the processor;

memory responsive to the processor;

a machine-readable medium responsive to the processor; and

a heuristic event handler in the machine-readable medium, wherein the heuristic event handler, when executed by the processor, enables the processor to perform operations comprising:

after a performance monitoring interrupt (PMI) was triggered by the heuristic event counter when the processor was executing a target program in user mode, and after the processor has automatically switched to kernel mode in response to the PMI, automatically performing preliminary analysis in kernel mode, without switching back to user mode, to determine whether heavyweight code analysis is warranted, wherein the preliminary analysis comprises:

obtaining an instruction pointer (IP) for the target program from the LBR buffer;

using the transaction hardware to determine whether the IP from the LBR buffer points to a readable page in the memory; and

determining that heavyweight code analysis is not warranted in response to a determination that the page pointed to by the IP from the LBR buffer is not readable.

2. A data processing system according to claim 1 , wherein the operation of using the transaction hardware to determine whether the IP from the LBR buffer points to a readable page in the memory comprises:

attempting to read from the page and using the transaction hardware to prevent the attempted read from causing a memory exception.

3. A data processing system according to claim 1 , wherein the heuristic event handler, when executed, performs further operations comprising:

in response to a determination that heavyweight code analysis is not warranted, automatically clearing the PMI and switching the processor from kernel mode to user mode, to enable the target program to resume execution.

4. A data processing system according to claim 1 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

determining that heavyweight analysis is not warranted, based on a branch mispredict bit from the LBR buffer.

5. A data processing system according to claim 1 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

using the transaction hardware in the processor to determine whether the IP from the LBR buffer points to a writable page in the memory; and

determining that heavyweight analysis is not warranted, based on a determination that the page pointed to by the IP from the LBR buffer is not writeable.

6. A data processing system according to claim 1 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

using the transaction hardware in the processor to determine whether the IP from the LBR buffer points to an executable page in the memory; and

determining that heavyweight analysis is not warranted, based on a determination that the page pointed to by the IP from the LBR buffer is not executable.

7. A data processing system according to claim 1 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

searching a region of memory used by the target program for a suspicious code signature without switching to user mode and without calling any operating system functions.

8. A data processing system according to claim 1 , wherein the heuristic event handler comprises a kernel driver.

9. A data processing system according to claim 1 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

automatically adding the IP from the LBR buffer to an IP whitelist, in response to determining that heavyweight code analysis is not warranted; and

subsequently using the IP whitelist to determine whether heavyweight code analysis is warranted after another PMI has been triggered.

10. An apparatus to enable a data processing system to detect control flow attacks, the apparatus comprising:

a non-transitory machine-readable medium; and

a heuristic event handler in the machine-readable medium, wherein the heuristic event handler, when executed by a processor of a data processing system, enables the data processing system to perform operations comprising:

after a heuristic event counter in a performance monitoring unit (PMU) in the processor has triggered a performance monitoring interrupt (PMI) when the processor was executing a target program in user mode, and after the processor has switched to kernel mode in response to the PMI, automatically performing preliminary analysis in kernel mode, without switching back to user mode, to determine whether heavyweight code analysis is warranted, wherein the preliminary analysis comprises:

obtaining an instruction pointer (IP) for the target program from a last branch record (LBR) buffer in the processor;

using transaction hardware in the processor to determine whether the IP from the LBR buffer points to a readable page in memory of the data processing system; and

determining that heavyweight code analysis is not warranted in response to a determination that the page pointed to by the IP from the LBR buffer is not readable.

11. An apparatus according to claim 10 , wherein the operation of using the transaction hardware to determine whether the IP from the LBR buffer points to a readable page in the memory comprises:

attempting to read from the page and using the transaction hardware to prevent the attempted read from causing a memory exception.

12. An apparatus according to claim 10 , wherein the heuristic event handler enables the data processing system to perform further operations comprising:

in response to a determination that heavyweight code analysis is not warranted, automatically clearing the PMI and switching the processor from kernel mode to user mode, to enable the target program to resume execution.

13. An apparatus according to claim 10 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

determining that heavyweight analysis is not warranted, based on a branch mispredict bit from the LBR buffer.

14. An apparatus according to claim 10 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

using the transaction hardware in the processor to determine whether the IP from the LBR buffer points to a writable page in the memory; and

determining that heavyweight analysis is not warranted, based on a determination that the page pointed to by the IP from the LBR buffer is not writeable.

15. An apparatus according to claim 10 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

using the transaction hardware in the processor to determine whether the IP from the LBR buffer points to an executable page in the memory; and

determining that heavyweight analysis is not warranted, based on a determination that the page pointed to by the IP from the LBR buffer is not executable.

16. An apparatus according to claim 10 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

searching a region of memory used by the target program for a suspicious code signature without switching to user mode and without calling any operating system functions.

17. An apparatus according to claim 10 , wherein the heuristic event handler comprises a kernel driver.

18. An apparatus according to claim 10 , wherein the preliminary analysis that the heuristic event handler enables the data processing system to perform in kernel mode further comprises:

automatically adding the IP from the LBR buffer to an IP whitelist, in response to determining that heavyweight code analysis is not warranted; and

subsequently using the IP whitelist to determine whether heavyweight code analysis is warranted after another PMI has been triggered.

19. A method for detecting control flow attacks, the method comprising:

after a heuristic event counter in a processor of a data processing system has triggered a performance monitoring interrupt (PMI) when the processor was executing a target program in user mode, and after the processor has switched to kernel mode in response to the PMI, automatically performing preliminary analysis in kernel mode, without switching back to user mode, to determine whether heavyweight code analysis is warranted, vs/herein the preliminary analysis comprises:

obtaining an instruction pointer (IP) for the target program from a last branch record (LBR) buffer in the processor;

using transaction hardware in the processor to determine whether the IP from LBR buffer points to a readable page in memory of the data processing system; and

determining that heavyweight code analysis is not warranted in response to a determination that the page pointed to by the IP from the LBR buffer is not readable.

20. A method according to claim 19 wherein the operation of using transaction hardware in the processor to determine whether the IP from the LBR buffer points to a readable page in the memory comprises:

attempting to read from the page and using the transaction hardware to prevent the attempted read from causing a memory exception.

21. A method according to claim 19 , further comprising:

in response to a determination that heavyweight code analysis is not warranted, automatically clearing the PMI, switching the processor from kernel mode to user mode, and then resuming execution of the target program; and

in response to a determining that heavyweight code analysis is warranted, automatically switching the processor from kernel mode to user mode and then using execution analysis software that is embedded in the target program to perform heavyweight code analysis in user mode.

22. A method according to claim 19 , wherein the preliminary analysis that is performed in kernel mode further comprises:

determining that heavyweight analysis is not warranted, based on a branch mispredict bit from the LBR buffer.

23. A method according to claim 19 , wherein the preliminary analysis that is performed in kernel mode further comprises:

using the transaction hardware in the processor to determine whether the IP from the LBR buffer points to a writable page in the memory; and

determining that heavyweight analysis is not warranted, based on a determination that the page pointed to by the IP from the LBR buffer is not writeable.

24. A method according to claim 19 , wherein the preliminary analysis that is performed in kernel mode further comprises:

using the transaction hardware in the processor to determine whether the IP from the LBR buffer points to an executable page in the memory; and

determining that heavyweight analysis is not warranted, based on a determination that the page pointed to by the IP from the LBR buffer is not executable.

25. A method according to claim 19 , wherein the preliminary analysis that is performed in kernel mode further comprises:

searching a region of memory used by the target program for a suspicious code signature without switching to user mode and without calling any operating system functions.