Methods and apparatuses for monitoring activities of virtual machines
View Patent ↗Embodiments of a method and apparatus for monitoring activity on a virtual machine are generally described herein. The activity may be monitored by a first hypervisor and the virtual machine may be controlled by a second hypervisor. In some embodiments, the method includes setting a breakpoint in a kernel function of the virtual machine. The method may further include generating a page fault, responsive to the virtual machine halting execution at the breakpoint, to cause the second hypervisor to page in contents of a memory location accessed by the kernel function. The method may further include inspecting the contents of the memory location to detect activity in the virtual machine.
1. A method comprising:
setting, by a first hypervisor, a breakpoint in a kernel function of a virtual machine that is controlled by a second hypervisor;
generating, by the first hypervisor, a page fault responsive to the virtual machine halting execution at the breakpoint to cause the second hypervisor to page in contents of a memory location accessed by the kernel function; and
inspecting, by the first hypervisor, the contents of the memory location to detect activity in the virtual machine;
wherein generating the page fault comprises:
retrieving, by the first hypervisor, a parameter of the kernel function indicating the memory location to be inspected; and
providing, by the first hypervisor, the parameter to the second hypervisor during generation of the page fault.
2. The method of claim 1 , further comprising:
accessing an operating system data structure to determine an address of the kernel function at which to set the breakpoint.
3. The method of claim 2 , further comprising:
selecting, from a plurality of virtual machines, the virtual machine in which to set the breakpoint based on pointers into an address translation table corresponding to each of the plurality of virtual machines.
4. The method of claim 3 , wherein a first of the virtual machines is based on a first operating system and a second of the virtual machines is based on a second operating system.
5. The method of claim 1 , further comprising:
determining that a page of memory contents to be inspected has been paged out by the second hypervisor; and
generating a second page fault to cause the second hypervisor to page in additional memory contents responsive to the determining.
6. The method of claim 1 , wherein the first hypervisor is of a first type and the second hypervisor is of a second type, the second type being different from the first type.
7. The method of claim 6 , wherein the first hypervisor is a bare-metal hypervisor and the second hypervisor is not a bare-metal hypervisor.
8. The method of claim 6 , wherein the first hypervisor is a first type of bare-metal hypervisor and the second hypervisor is a second type of bare-metal hypervisor.
9. The method of claim 3 , wherein:
the address translation table is an extended page table (EPT), and
the page fault indicates that an EPT entry is missing.
10. The method of claim 9 , wherein the page fault includes a VM Exit command.
11. A non-transitory computer-readable medium embodying a computer program, the computer program comprising instructions that, when executed by at least one processor, cause the at least one processor to:
use a first hypervisor to set a breakpoint in a kernel function of a virtual machine that is controlled by a second hypervisor;
use the first hypervisor to generate a page fault responsive to the virtual machine halting execution at the breakpoint to cause the second hypervisor to page in contents of a memory location accessed by the kernel function; and
use the first hypervisor to inspect the contents of the memory location to detect activity in the virtual machine;
wherein the instructions to cause the at least one processor to use the first hypervisor to generate the page fault comprise instructions that cause the at least one processor to use the first hypervisor to:
retrieve a parameter of the kernel function indicating the memory location to be inspected; and
provide the parameter to the second hypervisor during generation of the page fault.
12. The non-transitory computer-readable medium of claim 11 , wherein the page fault indicates that an extended page table (EPT) entry is missing.
13. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the at least one processor to:
select, from a plurality of virtual machines, the virtual machine in which to set the breakpoint based on pointers corresponding to each of the plurality of virtual machines.
14. The non-transitory computer-readable medium of claim 13 , wherein a first of the virtual machines is based on a first operating system and a second of the virtual machines is based on a second operating system.
15. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the at least one processor to:
determine that a page of memory contents to be inspected has been paged out by the second hypervisor; and
generate a second page fault to cause the second hypervisor to page in additional memory contents responsive to the determining.
16. An apparatus comprising:
a first hypervisor configured to:
set a breakpoint in a kernel function of a virtual machine that is controlled by a second hypervisor;
retrieve a parameter of the kernel function indicating a memory location to be inspected by the first hypervisor;
generate a page fault responsive to the virtual machine halting execution at the breakpoint to cause the second hypervisor to page in contents of the memory location; and
provide the parameter to the second hypervisor during generation of the page fault;
a memory configured to page-in, for inspection by the first hypervisor, the contents of the memory location that were paged-out by the second hypervisor; and
an address translation table configured to map virtual machine addresses to physical memory addresses.
17. The apparatus of claim 16 , wherein:
the page fault indicates that an entry is missing in the address translation table, and
the first hypervisor is further configured to access an operating system data structure to determine an address of the kernel function at which to set the breakpoint.
18. The apparatus of claim 17 , wherein:
the first hypervisor is further configured to select, from a plurality of virtual machines, the virtual machine in which to set the breakpoint based on pointers corresponding to each of the plurality of virtual machines; and
the page fault is a VM Exit command.
19. The apparatus of claim 17 , wherein a first of the virtual machines is based on a first operating system and a second of the virtual machines is based on a second operating system.
20. The apparatus of claim 16 , wherein the first hypervisor is further configured to:
determine that a page of memory contents to be inspected has been paged out by the second hypervisor; and
generate a second page fault to cause the second hypervisor to page in additional memory contents responsive to the determining.
21. A method comprising:
installing a first bare-metal hypervisor;
detecting, by the first bare-metal hypervisor, that a second hypervisor has initialized a virtual machine by examining a virtual memory table corresponding to the second hypervisor;
determining, by the first bare-metal hypervisor, addresses for locations of interest on the virtual machine; and
generating a page fault condition to cause the second hypervisor to provide memory contents for memory at the locations of interest, the page fault condition indicating that an entry is missing in the virtual memory table;
wherein generating the page fault condition comprises:
retrieving, by the first hypervisor, a parameter of a kernel function indicating a memory location among the locations of interest; and
providing, by the first hypervisor, the parameter to the second hypervisor during generation of the page fault.
22. The method of claim 21 , wherein generating the page fault condition comprises generating a virtual machine exit command, and wherein the method further comprises reading the memory contents responsive to receiving a notification that the second hypervisor has re-entered the virtual machine.