Exception handlers in a sandbox environment for malware detection

Inventor: Jason Neal Raber (Bellbrook, OH)

Assignee: Malwarebytes Inc.

G06F21/566 G06F21/53 G06F2221/033

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,797,676
App. No.: 17/411,833
Granted: Oct 24, 2023
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

An anti-malware application can emulate a suspicious program in a sandbox environment and retrieve any exception handlers the suspicious program attempts to register with the operation system. When the suspicious program triggers an exception, the anti-malware application can save a current context of the suspicious program being emulated. To emulate the handling of the exception, the anti-malware application can validate an exception handler chain including one or more exception handlers added by the suspicious program. The anti-malware application can then select and emulate an exception handler based on the saved context of the suspicious program at the time the exception was triggered. If the first exception handler is successful at resolving the exception, the anti-malware application can then save an updated post-exception context and continue emulation of the suspicious program based on the result of the first exception handler.

Claims (63)

1. A method comprising:

identifying, at a computing device, a suspicious program attempting to add an exception handler to an exception handler chain of the computing device,

in a sandbox environment that emulates the suspicious program, determining whether the suspicious program is a malicious program by:

generating a wrapped exception handler for the exception handler chain;

sequentially evaluating each exception handler in the exception handler chain with the wrapped exception handler by:

emulating an exception trigger for an exception handler in the exception handler chain;

evaluating the exception handler triggered by the emulated exception trigger;

responsive to evaluation of the exception handler indicating the suspicious program is non-malicious, evaluating a next exception handler in the exception handler chain; and

responsive to evaluation of the exception handler indicating the suspicious program is malicious, classifying the suspicious program as the malicious program.

2. The method of claim 1 , wherein evaluation of the exception handler indicates the suspicious program is non-malicious responsive to the suspicious program stopping execution in the sandbox environment after evaluation of the exception handler.

3. The method of claim 1 , wherein evaluation of the exception handler indicates the suspicious program is malicious when the suspicious program continues executing in the sandbox environment after evaluation of the exception handler.

4. The method of claim 1 , wherein evaluation of the exception handler further comprises saving a current emulation context of the suspicious program in the sandbox environment and evaluating the next exception handler in the exception handler chain comprises loading, before evaluating the next exception handler, the current emulation context for the suspicious program into the sandbox environment.

5. The method of claim 4 , wherein the emulation context comprises one or more of:

a current state of a processor stack in the sandbox environment,

a current state of a memory in the sandbox environment, or

one or more registers used by the suspicious program in the sandbox environment.

6. The method of claim 1 , wherein determining whether the suspicious program is the malicious program further comprises determining whether evaluation of the exception handler meets a success criteria.

7. The method of claim 1 , further comprising:

responsive to evaluation of the exception handler, saving a new emulation context of the suspicious program for the sandbox environment;

loading the new exception emulation context of the suspicious program into the sandbox environment; and

evaluating the next exception handler in the exception handler chain with the wrapped exception handler using the post-exception emulation context.

8. The method of claim 1 , further comprising:

responsive to classifying the suspicious program as the malicious program, removing the malicious program from the computing device.

9. The method of claim 1 , wherein evaluating the exception handler with the wrapped exception handler further comprises simulating a function executed by the exception handler in the sandbox environment when the exception handler is triggered by an exception trigger.

10. The method of claim 1 , wherein determining whether the suspicious program is the malicious program further comprises validating memory locations of exception handlers in the exception handler chain.

11. A non-transitory computer readable storage medium comprising instructions which, when executed by a processor, cause the processor to perform the steps of: