VMCloak: Toward a stealthy in-VM agent execution

In-VM agent programs are generally used for the convenience of VM monitor in malware analysis. To prevent malicious interference, the stealthy execution of an in-VM agent is desirable. Existing approaches for stealthy execution of the agent remains detectable if libraries or kernel code of the guest OS is contaminated by malware. Moreover, the lack of applicability to conventional executables limits the agent functionality. In this paper, VMCloak is proposed for the stealthy in-VM agent execution. Our scheme leverages the virtualization technology to perform the real-time binary instrumentation to conceal the fingerprints of an in-VM agent from potential detection. Both stealthiness and integrity of the agent are guaranteed even when the guest OS is compromised. The evaluation shows that VMCloak can cope with the applicability issues, allowing the in-VM agent to perform the same operations as that of ordinary executables.