REVIEW 4 major objections 5 minor 42 references
User-space library rootkits revisited: Are user-space detection mechanisms futile?
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A Linux library rootkit can make any user-space detector see a clean system.
desk verdict Useful, technically solid evasions against unhide, but the impossibility claim overreaches the evidence and is forced by the threat model. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is dynamic-library hooking through LD_PRELOAD, an environment variable that makes the Linux dynamic loader load a chosen library ahead of the normal ones, letting it intercept libc functions. The paper's particular leverage point is the execve hook: because the shell that launches the detector is itself hooked, the rootkit can intercept the detector's process creation and control everything downstream, including swapping the binary, scrubbing environment variables, filtering standard output through a pseudoterminal and a sed pipeline, entering new mount and PID namespaces with a private /proc, or running the detector under a ptrace tracer that rewrites getdents buffers in the detector's memory. This chain is what defeats statically linked binaries and direct-syscall detectors that avoid classic library hooking.
What would settle it
Find a hidden-process detector that runs only in user-space, is launched by the rootkit-controlled shell, and still reports the hidden PID by using side information the rootkit did not sanitize, such as timing differences, /proc/stat counters, or CPU performance counters. A single such detector would refute the paper's "cannot be done in user-space at all" claim; the experiments here provide no such detector.
Extended reading notes
Core claim
The paper's central claim, stated most strongly in the abstract, is that "the detection of user-space rootkits cannot be done in user-space at all," and that even a successful detection must be reported through a channel the rootkit cannot tamper with. The reason is not a missing technique in current tools but a structural asymmetry: "the anti-rootkit is executed by the rootkit," so the detector and the attacker run at the same privilege level while the attacker controls the detector's creation, arguments, environment, input/output, and the view of /proc it sees. The experiments show that minimal LD_PRELOAD hooks can hide a process from ps, top, and unhide's proc scan, and that an execve hook can rewrite the detector's binary, strip its environment, filter its output, move it into a new PID and mount namespace, or attach a ptrace tracer that rewrites directory entries returned by getdents even to a statically linked detector. The authors conclude that effort spent on pure user-space detection is unproductive and that future work must use kernel components or dedicated hardware for both detection and result delivery.
Load-bearing premise
In Section 1 assumptions 1, 2, and 4, restated in Section 6 as "the anti-rootkit is executed by the rootkit," the paper assumes the rootkit can recognize the detector and controls how the detector process is created, configured, and connected to the user; if the detector can be started from a trusted boot medium, run inside a hardware enclave, or receive kernel-signed output, the impossibility conclusion does not follow.
Editorial extensions
If this is right
- A user-space anti-rootkit that can be recognized and hooked is not a dependable defense against an advanced Linux library rootkit; the paper shows every unhide detection mode bypassed.
- Static linking and direct system calls do not restore trust, because an execve hook can substitute a malicious binary and a ptrace tracer can rewrite system call results.
- Anti-rootkit results cannot be trusted when printed through the same user-space environment the rootkit controls; a filter or namespace trick can alter what the user reads.
- Usable detection must therefore involve a kernel component, and the report must reach the user through a channel the rootkit cannot modify, such as the kernel console or a dedicated hardware display.
- The same conclusion applies to Linux systems with default dynamic linking in /bin and /sbin, where every command is vulnerable to hooking.
Reading between the lines
- A natural extension beyond the paper: the same impossibility argument should transfer to other dynamically linked Unix-like systems, because the structure, a detector launched through a hookable shell that inspects proc-like files, is not Linux-specific.
- The boundary of the claim is untested rather than disproved: cases with trusted-boot detectors, trusted execution environments, or kernel-mediated result delivery are outside the paper's model, and testing them would show how far the impossibility extends.
- Another implication the authors leave implicit is that the same execve, namespace, and ptrace techniques could manipulate not just anti-rootkits but any trusted user-space security agent that relies on user-space data sources.
- A testable extension: instrument the proof-of-concept rootkits with externally visible side effects, such as network flows or CPU counters, and check whether an out-of-band monitor outside the compromised environment can still identify the hidden process.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper revisits user-space library rootkits on Linux and asks whether user-space detection tools can be effective. It explains LD_PRELOAD-based hooking and process hiding through /proc, then presents a series of proof-of-concept experiments that evade the unhide anti-rootkit's process hiding checks (and one direct-syscall getdents detector) under a threat model in which the rootkit controls the user-space environment, can hook the shell, and can recognize the anti-rootkit binary. The authors conclude that detection of user-space rootkits cannot be done in user-space at all, and that scan results must be delivered through a channel that user-space components cannot tamper with.
Significance. The paper has real didactic and engineering value: it gives a clear account of LD_PRELOAD hooking, a useful catalogue of evasion techniques (library hooking, binary subversion, output filtering, double-personality execution, mount and PID namespace manipulation, and ptrace-based system call hooking), and honest discussion of why several common countermeasures fail under the stated assumptions. The experiments are concrete and the code snippets are, modulo minor formatting issues, plausible and instructive. However, the universal conclusion 'cannot be done in user-space at all' is not supported by the evidence presented. The experiments cover only unhide (standard and -ng) and one blog-style directory-listing detector, and every evasion depends on the rootkit being able to recognize the detector via isunhide/isdetector checks. The paper is best read as demonstrating that, under a favorable threat model, even unhide can be evaded; it does not demonstrate the impossibility of all user-space detection.
major comments (4)
- [Section 1, Assumption 4; Sections 5.2.1-5.3.2] The universal claim in the Abstract and in Section 6 that user-space detection 'cannot be done in user-space at all' is not supported by the experiments, because every evasion is keyed on the rootkit recognizing the detector: isunhide(pathname) in Sections 5.2.1, 5.2.2, 5.2.3, 5.3.1, and 5.3.2, and isdetector() in Section 5.4.1. If a detector uses a randomized filename, a packed or self-modifying payload with no recognizable signature, or is launched by a trusted parent outside the rootkit-controlled shell, the concrete evasions do not apply. Section 5.2.4 asserts that the rootkit can use 'behavior based detection' to recognize any detector, but no experiment or formal argument establishes that such recognition is unavoidable; this is the load-bearing point that must be proved or the claim must be weakened.
- [Sections 1 and 5; Section 7] The experimental coverage is too narrow for the conclusion drawn in Section 7 that 'investing effort in designing and developing user-space tools for detection without any help from the kernel is not worthwhile.' Only unhide and a single getdents-based detector are tested, and the ptrace experiment in Section 5.4.1 concerns file hiding rather than process hiding. The paper does not test other detection strategies, such as namespace-identity verification, direct block-device metadata parsing, or detection from a trusted launch environment, and it does not justify the assumption that unhide's detection logic is representative of the entire space of user-space detectors.
- [Section 6] The statement 'the main problem is that the anti-rootkit is executed by the rootkit' is an assumption inherited from the threat model in Section 1, not a demonstrated property of all deployment scenarios. The paper does not analyze detectors launched from initramfs, from signed boot environments, inside a TEE, or before the rootkit has installed its hooks. Consequently, the stronger claim in Section 6 that the disadvantage 'holds true for any system' is unsupported. The concrete evasions are conditional on the stated model; the impossibility conclusion is not.
- [Abstract and Section 7] The claim that 'the detection results must be communicated to the user with extreme caution' is well supported for the specific scenario in which results are displayed through user-space processes controlled by the rootkit, as demonstrated by the output-filtering and pseudoterminal experiments in Section 5.2.2. However, the conclusion in Section 7 that even kernel-side detection leaves communication as 'a highly delicate issue' is not demonstrated; the paper only shows that user-space output channels can be tampered with, and it does not analyze any kernel-level or hardware-level output channel. This part of the conclusion should be framed as a recommendation or an open problem rather than an established result.
minor comments (5)
- [Throughout] There are several typographical errors that should be corrected in a final version, including 'contemporany' in the Abstract, 'ouput' in Section 1.1, 'appart' in Section 5.3.2, and 'cyphering' in Section 5.4.2.
- [Section 3.1] The statement that 'all commands are vulnerable to hooking' is too strong, since statically linked binaries are immune to LD_PRELOAD hooking, as the paper itself notes in Section 5.1.2; the sentence should be qualified to dynamically linked commands.
- [Section 5.2] Figure 1 is referenced in the text on shell command execution, but no actual figure appears in the manuscript; either include the figure or remove the reference.
- [Sections 5.1-5.4] The code snippets contain spacing artifacts (e.g., 'un si gne d', 'co nt inu e', 'i s d e t e c t o r') that appear to result from LaTeX or PDF extraction; these should be cleaned so the code can be copied and tested directly.
- [Section 6] The paper's abstract says 'cannot be done in user-space at all,' while Section 6 says an advanced adversary 'is capable of creating a user-space rootkit that is virtually undetectable in user-space.' These are different claims; the abstract should be aligned with the evidence and use the weaker, conditional formulation.
Circularity Check
No significant circularity: the impossibility claim is conditional on a stated threat model, not derived from itself.
full rationale
After walking the claimed derivation chain, no circular step is present. The paper's Section 1 assumptions (rootkit controls user-space, hooks libraries, can identify the anti-rootkit) are stated premises of an adversarial model, not conclusions derived from the experiments. The experiments in Section 5 provide concrete, independent engineering demonstrations that under those premises unhide can be evaded; the evasions do not assume the impossibility conclusion. The broad abstract phrase 'cannot be done in user-space at all' is stronger than what the model supports, but overgeneralization from a stated threat model is a soundness/scope concern, not a definitional or self-citation circularity. There are no fitted parameters renamed as predictions, no load-bearing self-citations, and no imported uniqueness theorem; the countermeasure discussion explicitly acknowledges channels outside user-space (kernel components, dedicated hardware), which further shows the conclusion is not identical to its assumptions. Score 0.
Assumptions & free parameters
assumptions (7)
- domain assumption The rootkit can replace system binaries and intercept dynamic libraries via LD_PRELOAD or equivalent hooking (Section 1, assumption 1).
- domain assumption The rootkit can identify the anti-rootkit binary by data or metadata (Section 1, assumption 4).
- ad hoc to paper unhide's detection logic is representative of standard user-space anti-rootkit detection (Section 1 and Section 5).
- domain assumption System binaries in the target distribution are dynamically linked and therefore hookable (Section 3.1, Ubuntu 24.04 example).
- domain assumption Kernel-space components are trusted, but the anti-rootkit cannot run privileged code (Section 1, assumption 3).
- domain assumption ptrace with PTRACE_SYSCALL allows the tracer to observe and rewrite the tracee's syscall arguments and results (Section 5.4.1).
- domain assumption unshare and mount can create private namespaces, including a fake /proc, for a child process without affecting other processes (Section 5.3).
Cite this review
Pith. "Pith review of User-space library rootkits revisited: Are user-space detection mechanisms futile?." pith.science (2026). https://pith.science/paper/SA2WKXJ2
@misc{pith2026250607827,
author = {Pith},
title = {Pith review of: User-space library rootkits revisited: Are user-space detection mechanisms futile?},
year = {2026},
howpublished = {\url{https://pith.science/paper/SA2WKXJ2}},
note = {Machine review of arXiv:2506.07827}
}
read the original abstract
The kind of malware designed to conceal malicious system resources (e.g. processes, network connections, files, etc.) is commonly referred to as a rootkit. This kind of malware represents a significant threat in contemporany systems. Despite the existence of kernel-space rootkits (i.e. rootkits that infect the operating system kernel), user-space rootkits (i.e. rootkits that infect the user-space operating system tools, commands and libraries) continue to pose a significant danger. However, kernel-space rootkits attract all the attention, implicitly assuming that user-space rootkits (malware that is still in existence) are easily detectable by well-known user-space tools that look for anomalies. The primary objective of this work is to answer the following question: Is detecting user-space rootkits with user-space tools futile? Contrary to the prevailing view that considers it effective, we argue that the detection of user-space rootkits cannot be done in user-space at all. Moreover, the detection results must be communicated to the user with extreme caution. To support this claim, we conducted different experiments focusing on process concealing in Linux systems. In these experiments, we evade the detection mechanisms widely accepted as the standard solution for this type of user-space malware, bypassing the most popular open source anti-rootkit tool for process hiding. This manuscript describes the classical approach to build user-space library rootkits, the traditional detection mechanisms, and different evasion techniques (it also includes understandable code snippets and examples). In addition, it offers some guidelines to implement new detection tools and improve the existing ones to the extent possible.
Figures
Reference graph
Works this paper leans on
-
[1]
G. Hoglund and J. Butler,Rootkits: Subverting the Windows Kernel. Addison-Wesley Professional, 2005, ch. one: Leave No Trace
work page 2005
-
[2]
M. A. Davis, S. M. Bodmer, and A. LeMasters,Hacking Exposed: Mal- ware and Rootkits Secrets and Solutions. McGraw-Hill Education, 2009, ch. three: user-mode rootkits
work page 2009
-
[3]
Subverting vista kernel for fun and profit
J. Rutkowska, “Subverting vista kernel for fun and profit.” [Online]. Available: http://blackhat.com/presentations/bh-usa-06/ BH-US-06-Rutkowska.pdf
-
[4]
Subvirt: implementing malware with vir- tual machines,
S. T. King and P. M. Chen, “Subvirt: implementing malware with vir- tual machines,” in2006 IEEE Symposium on Security and Privacy (S P’06), 2006, pp. 14 pp.–327
work page 2006
-
[5]
Covirt (a virtual-machine based rootkit)
“Covirt (a virtual-machine based rootkit).” [Online]. Available: https://github.com/Nadharm/CoVirt
-
[6]
Linux on-the-fly kernel patching without lkm,
Sd and Devik, “Linux on-the-fly kernel patching without lkm,” inPhrack Volume 0x0b, Issue 0x3a, 2001. [Online]. Available: https://phrack.org/issues/58/7 11https://www.bing.com/chat 41
work page 2001
-
[7]
“Triplecross.” [Online]. Available: https://github.com/h3xduck/ TripleCross?tab=readme-ov-file
-
[8]
The hidden threat: Analysis of linux rootkit techniques and limitations of current detection tools,
J. St¨ uhn, J.-N. Hilgert, and M. Lambertz, “The hidden threat: Analysis of linux rootkit techniques and limitations of current detection tools,”Digital Threats, vol. 5, no. 3, Oct. 2024. [Online]. Available: https://doi.org/10.1145/3688808
Show all 42 references
-
[9]
Unhide git repository
“Unhide git repository.” [Online]. Available: https://github.com/ YJesus/Unhide
-
[10]
Unhide: the open source forensic tool
“Unhide: the open source forensic tool.” [Online]. Available: https://www.unhide-forensics.info/?Linux
-
[11]
Unhide-ng (next generation) git repository
“Unhide-ng (next generation) git repository.” [Online]. Available: https://github.com/YJesus/Unhide-NG
-
[12]
Kong,Designing BSD Rootkits
J. Kong,Designing BSD Rootkits. USA: No Starch Press, 2007
2007
-
[13]
Blunden,The Rootkit Arsenal: Escape and Evasion in the Dark Cor- ners of the System
B. Blunden,The Rootkit Arsenal: Escape and Evasion in the Dark Cor- ners of the System. Jones and Bartlett Learning, 2009
2009
-
[14]
Efficient and trusted detection of rootkit in iot devices via offline profiling and online monitoring,
X. Jiang, M. Lora, and S. Chattopadhyay, “Efficient and trusted detection of rootkit in iot devices via offline profiling and online monitoring,” inProceedings of the 2020 on Great Lakes Symposium on VLSI, ser. GLSVLSI ’20. New York, NY, USA: Association for Computing Machiner...
2020
-
[15]
Multi-aspect profiling of kernel rootkit behavior,
R. Riley, X. Jiang, and D. Xu, “Multi-aspect profiling of kernel rootkit behavior,” inProceedings of the 4th ACM European Conference on Computer Systems, ser. EuroSys ’09. New York, NY, USA: Association for Computing Machinery, 2009, p. 47–60. [Online]. Available: https://doi....
2009
-
[16]
An online cross view difference and behavior based kernel rootkit detector,
C. Mahapatra and S. Selvakumar, “An online cross view difference and behavior based kernel rootkit detector,”SIGSOFT Softw. Eng. Notes, vol. 36, no. 4, p. 1–9, Aug. 2011. [Online]. Available: https://doi.org/10.1145/1988997.1989022
2011
-
[17]
On the detection of kernel-level rootkits using hardware performance counters,
B. Singh, D. Evtyushkin, J. Elwell, R. Riley, and I. Cervesato, “On the detection of kernel-level rootkits using hardware performance counters,” inProceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, ser. ASIA CCS ’17. New York, NY, USA: 42 A...
2017
-
[18]
Position paper: Consider hardware-enhanced defenses for rootkit attacks,
G. Hu, T. Zhang, and R. B. Lee, “Position paper: Consider hardware-enhanced defenses for rootkit attacks,” inProceedings of the 9th International Workshop on Hardware and Architectural Support for Security and Privacy, ser. HASP ’20. New York, NY, USA: Association for Computin...
2021
-
[19]
Advance kernel rootkit de- tection: Survey,
S. Suresh Kumar and T. SudalaiMuthu, “Advance kernel rootkit de- tection: Survey,” in2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), 2023, pp. 944–948
2023
-
[20]
Rootkit modeling and ex- periments under linux,
E. Lacombe, F. Raynal, and V. Nicomette, “Rootkit modeling and ex- periments under linux,”Journal in Computer Virology, vol. 4, pp. 137– 157, 05 2008
2008
-
[21]
Rootkit detection mechanism: A survey,
J. Joy, A. John, and J. Joy, “Rootkit detection mechanism: A survey,” inAdvances in Parallel Distributed Computing, D. Nagamalai, E. Re- nault, and M. Dhanuskodi, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 366–374
2011
-
[22]
A framework for prototyping and testing data-only rootkit attacks,
R. Riley, “A framework for prototyping and testing data-only rootkit attacks,”Computers and Security, vol. 37, pp. 62–71, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0167404813000825
2013
-
[23]
The design of the simple smm rootkit,
M. Szaknis and K. Szczypiorski, “The design of the simple smm rootkit,” inProceedings of the 2022 9th International Conference on Wireless Communication and Sensor Networks, ser. icWCSN ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 47–56. [Online]. Avai...
2022
-
[24]
The research on rootkit for information system classified protection,
Z. Tian, B. Wang, Z. Zhou, and H. Zhang, “The research on rootkit for information system classified protection,” in2011 International Confer- ence on Computer Science and Service System (CSSS), 2011, pp. 890– 893
2011
-
[25]
Kernel rootkit experiences,
stealth, “Kernel rootkit experiences,” inPhrack Volume 0x0b, Issue 0x3d, 2003. [Online]. Available: https://phrack.org/issues/61/14 43
2003
-
[26]
Execution path analysis: finding kernel based rootkits,
J. K. Rutkowski, “Execution path analysis: finding kernel based rootkits,” inPhrack Volume 0x0b, Issue 0x3b, 2002. [Online]. Available: https://phrack.org/issues/59/10
2002
-
[27]
Finding hidden kernel modules (extrem way reborn): 20 years later,
g1inko, “Finding hidden kernel modules (extrem way reborn): 20 years later,” inPhrack Volume 0x10, Issue 0x47, 2024. [Online]. Available: https://phrack.org/issues/71/12
2024
-
[28]
Shared library redirection techniques,
halflife, “Shared library redirection techniques,” inPhrack Volume 7, Issue 51, 1997. [Online]. Available: https://phrack.org/issues/51/8
1997
-
[29]
He compels secure shells,
DangerMouse, “He compels secure shells,” inPhrack Volume 0x0f, Issue 0x45, 2016. [Online]. Available: https://phrack.org/issues/69/4
2016
-
[30]
The art of linux kernel rootkits,
Matheuzsec and Humzak711, “The art of linux kernel rootkits,” in tmp.0ut Volume 4, 2025
2025
-
[31]
Bypassing ldpreload rootkits is easy,
“Bypassing ldpreload rootkits is easy,” accessed on 05.23.2025. [Online]. Available: https://matheuzsecurity.github.io/hacking/ bypass-userland-hooks/
2025
-
[32]
In-depth study of linux rootkits: Evolution, detection, and defense
S. Berger, “In-depth study of linux rootkits: Evolution, detection, and defense.” 2025 FIRST Technical Colloquium, Amsterdam, 2025
2025
-
[33]
Malware memory analysis of the jynx2 linux rootkit: Investigating a publicly available linux rootkit using the volatility memory analy- sis framework,
“Malware memory analysis of the jynx2 linux rootkit: Investigating a publicly available linux rootkit using the volatility memory analy- sis framework,” Defence Research and Development Canada, Scientific Report DRDC-RDDC-2014-R176, 2014
2014
-
[34]
Shared library call redirection via elf plt infection,
S. Cesare, “Shared library call redirection via elf plt infection,” inPhrack Volume 0xa Issue 0x38, 2000. [Online]. Available: https://phrack.org/issues/71/12
2000
-
[35]
How detect a ldpreload rootkit and hide from ldd and /proc,
“How detect a ldpreload rootkit and hide from ldd and /proc,” accessed on 06.03.2025. [Online]. Available: https://matheuzsecurity.github.io/ hacking/ldpreload-rootkit/
2025
-
[36]
Snoopy command logger
“Snoopy command logger.” [Online]. Available: https://github.com/ a2o/snoopy/
-
[37]
zpoline: a system call hook mechanism based on binary rewriting,
K. Yasukata, H. Tazaki, and P.-L. Aublin, “zpoline: a system call hook mechanism based on binary rewriting,” in2023 USENIX Annual Tech- nical Conference (USENIX ATC 23), 2023. 44
2023
-
[38]
Kerrisk,The Linux Programming Interface: A Linux and UNIX Sys- tem Programming Handbook, 1st ed
M. Kerrisk,The Linux Programming Interface: A Linux and UNIX Sys- tem Programming Handbook, 1st ed. USA: No Starch Press, 2010, ch. 64: Pseudoterminals
2010
-
[39]
The use of name spaces in plan 9,
R. Pike, D. Presotto, K. Thompson, H. Trickey, and P. Winterbottom, “The use of name spaces in plan 9,” inProceedings of the 5th Workshop on ACM SIGOPS European Workshop: Models and Paradigms for Distributed Systems Structuring, ser. EW 5. New York, NY, USA: Association for Co...
1992
-
[40]
The ptrace anti Re trick
“The ptrace anti Re trick.” [Online]. Available: https://hkopp.github. io/2023/08/the-ptrace-anti-re-trick
2023
-
[41]
The linux kernel documentation: Syscall user dispatch
“The linux kernel documentation: Syscall user dispatch.” [Online]. Available: https://docs.kernel.org/admin-guide/syscall-user-dispatch. html
-
[42]
The linux kernel documentation: ftrace - function tracer
“The linux kernel documentation: ftrace - function tracer.” [Online]. Available: https://www.kernel.org/doc/html/v5.0/trace/ftrace.html 45
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.