System and method of fileless malware detection and non-transitory computer readable medium
The present disclosure provides a system and a method of fileless malware detection, and the method of the fileless malware detection includes steps as follows. The execution of the writable section in the memory is intercepted; the executable code corresponding to the execution is extracted from the writable section; whether the executable code is malicious is analyzed.
1. A system of fileless malware detection, and the system comprising:
a memory; and
a processor electrically connected to the memory, and the processor configured to execute a check program for:
intercepting an execution of a writable section in the memory;
extracting an executable code corresponding to the execution from the writable section;
analyzing whether the executable code is malicious;
when an instruction is the execution, checking whether a NX bit (No execute bit) in a PTE (Page Table Entry) is 1;
when the NX bit is checked as 1 , triggering a page fault exception, and then determining whether a virtual memory area corresponding to the instruction is writable in response to that the page fault exception is triggered;
determining whether the virtual memory area is marked as executable in response to determining that the virtual memory area is writable; and
restoring the NX bit to 0 so as to restore the PTE in a normal state in response to determining that the virtual memory area is marked as executable, packaging a program code corresponding to the instruction from the virtual memory area into an ELF (Executable and Linkable Format) File to be scanned through a virus scanning software to generate a scan result, and determining whether the instruction can be executed according to the scan result.
2. The system of claim 1 , wherein the processor is configured to execute the check program for:
when the NX bit is checked as 0, allowing the execution.
3. The system of claim 1 , wherein the processor is configured to execute the check program for:
when the virtual memory area is not writable, terminating the execution.
4. The system of claim 1 , wherein the processor is configured to execute the check program for:
when receiving a system call as a default check item, checking a position of a file corresponding to the system call to determine whether the file is in the memory; and
when the file is in the memory, scanning the file and determining whether the file can be executed according to a scan result.
5. A method of fileless malware detection, and the method comprising:
intercepting an execution of a writable section in a memory;
extracting an executable code corresponding to the execution from the writable section;
analyzing whether the executable code is malicious;
when an instruction is the execution, checking whether a NX bit in a PTE is 1;
when the NX bit is checked as 1 , triggering a page fault exception, and then determining whether a virtual memory area corresponding to the instruction is writable in response to that the page fault exception is triggered;
determining whether the virtual memory area is marked as executable in response to determining that the virtual memory area is writable; and
restoring the NX bit to 0 so as to restore the PTE in a normal state in response to determining that the virtual memory area is marked as executable, packaging a program code corresponding to the instruction from the virtual memory area into an ELF File to be scanned through a virus scanning software to generate a scan result, and determining whether the instruction can be executed according to the scan result.
6. The method of claim 5 , further comprising:
when the NX bit is checked as 0, allowing the execution.
7. The method of claim 5 , further comprising:
when the virtual memory area is not writable, terminating the execution.
8. The method of claim 5 , further comprising:
when receiving a system call as a default check item, checking a position of a file corresponding to the system call to determine whether the file is in the memory; and
when the file is in the memory, scanning the file and determining whether the file can be executed according to a scan result.
9. A non-transitory computer readable medium to store a plurality of instructions for commanding a computer to execute a method of fileless malware detection, and the method comprising:
intercepting an execution of a writable section in a memory;
extracting an executable code corresponding to the execution from the writable section;
analyzing whether the executable code is malicious;
when an instruction is the execution, checking whether a NX bit in a PTE is 1;
when the NX bit is checked as 1, triggering a page fault exception, and then determining whether a virtual memory area corresponding to the instruction is writable in response to that the page fault exception is triggered;
determining whether the virtual memory area is marked as executable in response to determining that the virtual memory area is writable; and
restoring the NX bit to 0 so as to restore the PTE in a normal state in response to determining that the virtual memory area is marked as executable, packaging a program code corresponding to the instruction from the virtual memory area into an ELF File to be scanned through a virus scanning software to generate a scan result, and determining whether the instruction can be executed according to the scan result.
10. The non-transitory computer readable medium of claim 9 , wherein the method further comprises:
when the NX bit is checked as 0, allowing the execution.
11. The non-transitory computer readable medium of claim 9 , wherein the method further comprises:
when the virtual memory area is not writable, terminating the execution.
12. The non-transitory computer readable medium of claim 9 , wherein the method further comprises:
when receiving a system call as a default check item, checking a position of a file corresponding to the system call to determine whether the file is in the memory; and
when the file is in the memory, scanning the file and determining whether the file can be executed according to a scan result.