NanoHook: An Efficient System Call Hooking Technique with One-Byte Invasive
摘要
System calls serve as the primary interface between the operating system kernel and user-space programs and can be hooked to trace, analyze, and modify program behavior. Minimizing the overhead introduced by system call hooking techniques is crucial to prevent performance degradation of the hooked program. Moreover, modifications to the process instructions by the hooking technique should be minimal to avoid unintended side effects. However, existing techniques struggle to balance these requirements. Techniques such as syscall user dispatch (SUD) and ptrace avoid modifying process instructions but introduce substantial overhead, whereas other low-overhead techniques often require modifying multiple bytes or instructions to redirect the system call to the hook function. This paper introduces NanoHook, an innovative system call hooking technique tailored for 32-bit programs. NanoHook leverages the specific jump behavior of 32-bit programs during system calls. This approach only requires binary rewriting of one byte of a particular instruction to hook system calls with minimal overhead. Experimental results demonstrate that NanoHook reduces the overhead, defined as the additional time required to execute a system call after introducing the hooking technique, by a factor of 6.7 to 13, 335.6 compared to existing techniques. When applied to Redis, NanoHook incurs only a 4% performance loss, significantly lower than the 8.3% to 94.85% performance degradation observed with other techniques.