REVIEW 2 major objections 5 minor 42 references
A Measurement Study on the Adoption of Pledges and Unveils in the OpenBSD Operating System
T0 review · 2 major / 5 minor · reviewed 2026-07-12 · grok-4.5
Pith's one-line read OpenBSD has steadily adopted pledge and unveil sandboxing across its core userland, showing system-call minimization need not be as hard as the literature claims.
desk verdict Clean first longitudinal count of pledge/unveil inside OpenBSD’s own tree; solid data release, modest claims, already-flagged operationalization limits. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Longitudinal counts of pledge and unveil call sites (and their first-parameter promises or permission flags) extracted by simple string matching over .c files in successive OpenBSD source trees, treating each sub-directory as one program or library.
What would settle it
A re-parse of the same trees that recovers substantially fewer live call sites (e.g., after expanding macros or discarding dead or test-only code) or a developer survey showing that high call counts coexist with frequent pledge/unveil-related debugging pain or reverts would collapse the “not troublesome” claim.
Extended reading notes
Core claim
Across nineteen OpenBSD releases the number of pledge and unveil invocations grew continuously and roughly linearly; many programs already used the calls before they shipped in official releases, especially those installed under /usr/bin and /usr/sbin, while program size (proxied by C-file count) showed little correlation with call counts and the full set of promise and permission flags was exercised rather than only a few coarse options.
Load-bearing premise
That counting string occurrences of “pledge(” and “unveil(” inside selected .c files, and treating each sub-directory as a single program, accurately reflects successful, low-friction adoption rather than incomplete, conditional, or high-maintenance uses.
Editorial extensions
If this is right
- Simple, source-level sandbox primitives can achieve near-universal coverage inside a complete operating-system userland without requiring large-program complexity or size.
- Linear growth of such call sites can serve as a coarse but usable heuristic for tracking security-feature adoption over multi-year release histories.
- The same measurement approach can be applied to other OS-specific sandbox APIs to test whether adoption difficulty is inherent or design-dependent.
- Wholesale use of fine-grained promises and path permissions implies that the design of pledge/unveil is flexible enough for both trivial utilities and larger daemons.
Reading between the lines
- If the observed wholesale adoption is real, other open-source kernels may gain more traction by offering equally simple, source-embedded primitives rather than powerful but complex filter toolboxes.
- The lack of size correlation suggests that maintenance cost, not program scale, is the real barrier; measuring commit reverts or code-churn around pledge/unveil sites would be a direct next test.
- POSIX-level standardization of a minimal pledge-like interface could transfer the observed adoption pattern beyond the BSD family.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a longitudinal measurement of the adoption of the OpenBSD pledge and unveil system calls across 19 releases (5.9–7.7, 2016–2025). Using string matching on .c files under selected source-tree directories, the authors count invocations and their first (or second) parameters, treat each sub-directory as one program/library, and report (i) substantial pre-release adoption followed by roughly linear growth to >700 pledge and ~300 unveil calls, (ii) concentration in /usr/bin and /usr/sbin, (iii) weak Pearson correlations between call counts and a C-file-count size proxy (0.332 / 0.052), and (iv) frequent use of stdio/rpath-style promises and r-permission unveils while still exercising the broader parameter space. From these descriptive results the authors conclude that system-call minimization and sandboxing need not be as troublesome to adopt as prior literature has suggested.
Significance. The work supplies the first multi-release quantitative picture of pledge/unveil adoption inside OpenBSD’s own userland, complementing earlier interview and package-snapshot studies. The publicly released dataset and transparent parsing method make the counts reproducible and falsifiable. Even though the inference about “not troublesome” is modest and caveated, the descriptive facts themselves (wholesale continuous growth, weak size correlation, parameter usage patterns) are useful empirical anchors for the software-evolution and OS-security communities. The paper therefore contributes a solid, if limited, measurement baseline rather than a causal claim.
major comments (2)
- Section V (and the abstract) draw the inference that adoption “is not necessarily as troublesome as has often been discussed.” The manuscript itself notes (II-C, VI) that maintenance effort, debugging cost, and reverts are unmeasured. The observed growth is consistent with low friction but does not by itself demonstrate it; a single sentence that more tightly bounds the claim to “call-count growth and coverage are compatible with relatively low adoption friction” would keep the conclusion proportionate to the evidence.
- Section III-B operationalizes “program/library” as a source sub-directory and size as the count of .c files. Both choices are acknowledged as imperfect, yet they underwrite the ranking in Fig. 3 and the correlations in Fig. 4. A short sensitivity check (e.g., excluding multi-binary directories such as usr.bin/openssl, or reporting LOC instead of file count for a subset) would strengthen confidence that the weak-correlation result is not an artifact of the proxy.
minor comments (5)
- Fig. 1 caption and surrounding text call the linear fit a “coarse but sensible heuristic”; the same phrasing should appear in the abstract so readers do not over-interpret the trend line.
- Section III-A: the exclusion of distrib/gnu/regress/sys is reasonable, but a one-sentence justification that these trees are not part of the “core userland” under study would help readers unfamiliar with the OpenBSD layout.
- Section IV-C / Fig. 5: five undocumented promises were added ad hoc; listing them (or noting that they are rare) would improve reproducibility.
- Typographical consistency: “pledge” and “unveil” are sometimes italicized, sometimes not; a uniform convention would improve readability.
- References [3] and [4] are central to the motivation; ensuring the final versions (or arXiv identifiers) are cited once they appear would keep the literature trail current.
Circularity Check
No significant circularity: pure observational measurement study with no fitted-input-as-prediction or self-definitional steps.
full rationale
This is a longitudinal software-measurement paper. Its central claims rest on counts of the strings “pledge(” and “unveil(” extracted from archived OpenBSD source trees across 19 releases, plus simple descriptive statistics (medians, Pearson correlations, top-k rankings, and a visual linear trend). No parameter is fitted to one subset of the data and then re-presented as a prediction of a related quantity; the linear trend is explicitly labeled “a coarse but sensible heuristic,” not a derived law. No uniqueness theorem, ansatz, or load-bearing result is imported from the authors’ own prior work; the few self-citations are ordinary background or data-release references. The operationalization (sub-directory = program, C-file count = size proxy) is a measurement choice, not a circular definition of the adoption rates themselves. Consequently the derivation chain is self-contained against external source archives and contains none of the six circularity patterns.
Assumptions & free parameters
assumptions (3)
- domain assumption Occurrence of the character strings “pledge(” or “unveil(” inside a .c file constitutes a genuine, intentional use of the system call.
- ad hoc to paper Each sub-directory under the selected root directories corresponds to one logical program or library for counting purposes.
- domain assumption Number of .c files is an adequate proxy for software size when testing correlation with call counts.
Cite this review
Pith. "Pith review of A Measurement Study on the Adoption of Pledges and Unveils in the OpenBSD Operating System." pith.science (2026). https://pith.science/paper/MEMADKYD
@misc{pith2026260703056,
author = {Pith},
title = {Pith review of: A Measurement Study on the Adoption of Pledges and Unveils in the OpenBSD Operating System},
year = {2026},
howpublished = {\url{https://pith.science/paper/MEMADKYD}},
note = {Machine review of arXiv:2607.03056}
}
read the original abstract
The paper presents a longitudinal measurement study on the adoption of the pledge and unveil system calls in OpenBSD. These system calls are used to sandbox programs and libraries. Given a dataset covering 19 releases, many programs and libraries were modified to use the system calls already before their introductions in official releases. The adoption rates have also steadily grown; a linear trend provides a coarse but sensible heuristic. Although particularly programs residing in /usr/bin and /usr/sbin have been modified to use the system calls, the sizes of programs and libraries do not correlate well with the amounts of pledge and unveil system calls invoked. Regarding the pledges made, standard input and output operations have frequently been requested, although the full fine-grained arsenal offered by pledge has generally been utilized in OpenBSD. The same observation is seen in that particularly read operations to given paths have frequently been unveiled. All in all, the measurement results indicate that the adoption of system call minimization and sandboxing techniques is not necessarily as troublesome as has often been discussed in the literature.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Tunney: Porting OpenBSD pledge() to Linux,
J. Corbet, “Tunney: Porting OpenBSD pledge() to Linux,” 2022, Linux Weekly News (LWN), available online in October 2025: https://lwn.net/ Articles/901264/
2022
-
[2]
OpenBSD’s unveil(),
——, “OpenBSD’s unveil(),” 2018, Linux Weekly News (LWN), avail- able online in October 2025: https://lwn.net/Articles/767137/
2018
-
[3]
Sandboxing Adoption in Open Source Ecosystems,
M. Alhindi and J. Hallett, “Sandboxing Adoption in Open Source Ecosystems,” inProceedings of the 12th ACM/IEEE International Work- shop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS 2024). Lisbon: ACM, 2024, pp. 13–20
2024
-
[4]
Playing in the Sandbox: A Study on the Usability of Seccomp,
——, “Playing in the Sandbox: A Study on the Usability of Seccomp,” 2025, Archived manuscript, available online in October: https://arxiv. org/abs/2506.10234
arXiv 2025
-
[5]
E. S. Raymond,Basics of the Unix Philosophy: The Art of Unix Programming. London: Addison-Wesley, 2003, referenced in October 2025 from http://www.catb.org/esr/writings/taoup/html/ch01s06.html
2003
-
[6]
Eliminating Vulnerabilities by Disabling Unwanted Functionality in Binary Programs,
M. Mansouri, J. Xu, and G. Portokalidis, “Eliminating Vulnerabilities by Disabling Unwanted Functionality in Binary Programs,” inProceedings of the 2023 ACM Asia Conference on Computer and Communications Security (ASIA CCS 2023). Melbourne: ACM, 2023, pp. 259–273
2023
-
[7]
An Alignment Between the CRA’s Essential Requirements and the ATT&CK ®’s Mitigations,
J. Ruohonen, E.-Y . Kang, and Q. Ramadan, “An Alignment Between the CRA’s Essential Requirements and the ATT&CK ®’s Mitigations,” in Proceedings of the IEEE 33rd International Requirements Engineering Conference Workshops (REW 2025). Valencia: IEEE, 2025
2025
-
[8]
Attack Surface Definitions: A Systematic Literature Review,
C. Theisen, N. Munaiah, M. Al-Zyoud, J. C. Carver, A. Meneely, and L. William, “Attack Surface Definitions: A Systematic Literature Review,”Information and Software Technology, vol. 104, pp. 94–103, 2018
2018
Show all 42 references
-
[9]
A Comparison of Unix Sandboxing Techniques,
J. Anderson, “A Comparison of Unix Sandboxing Techniques,” The FreeBSD Journal, pp. 16–25, 2017, available online in Oc- tober 2025: https://freebsdfoundation.org/wp-content/uploads/2017/10/ A-Comparison-of-Unix-Sandboxing-Techniques.pdf
2017
-
[10]
Operating System Enhancements to Prevent the Misuse of System Calls,
M. Bernaschi, E. Gabrielli, and L. V . Mancini, “Operating System Enhancements to Prevent the Misuse of System Calls,” inProceedings of the 7th ACM conference on Computer and Communications Security (CCS 2000). Athens: ACM, 2000, pp. 174–183
2000
-
[11]
Confine: Automated System Call Policy Generation for Container Attack Surface Reduction,
S. Ghavamnia, T. Palit, A. Benameur, and M. Polychronakis, “Confine: Automated System Call Policy Generation for Container Attack Surface Reduction,” inProceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020). Virtual event: U...
2020
-
[12]
Optimus: Association-Based Dy- namic System Call Filtering for Container Attack Surface Reduction,
S. Yang, B. B. Kang, and J. Nam, “Optimus: Association-Based Dy- namic System Call Filtering for Container Attack Surface Reduction,” Journal of Cloud Computing, vol. 13, pp. 1–22, 2024
2024
-
[13]
Shrinking the Kernel Attack Surface Through Static and Dynamic Syscall Limitation,
D. Zhan, Z. Yu, X. Yu, H. Zhang, and L. Ye, “Shrinking the Kernel Attack Surface Through Static and Dynamic Syscall Limitation,”IEEE Transactions on Services Computing, vol. 16, no. 2, pp. 1431–1443, 2023
2023
-
[14]
Improving Host Security with System Call Policies,
N. Provos, “Improving Host Security with System Call Policies,” in Proceedings of the 12th USENIX Security Symposium. Washington: USENIX, 2003
2003
-
[15]
Linux seccomp(2) vs OpenBSD pledge(2),
G. Bechis, “Linux seccomp(2) vs OpenBSD pledge(2),” 2017, Open Source Summit Europe, Prague, available online in October 2025: https: //www.paclan.it/events/seccomp-pledge-oss2017.pdf
2017
-
[16]
Capsicum – lightweight OS capability and sandbox frame- work,
FreeBSD, “Capsicum – lightweight OS capability and sandbox frame- work,” 2024, FreeBSD Manual Pages, available online in October 2025: https://man.freebsd.org/cgi/man.cgi?capsicum(4)
2024
-
[17]
Capsicum: Practical Capabilities for UNIX,
R. N. M. Watson, J. Anderson, B. Laurie, and K. Kennaway, “Capsicum: Practical Capabilities for UNIX,” inProceedings of the 19th USENIX Security Symposium. Washington: USENIX, 2010
2010
-
[18]
hack – Exploring The Dungeons of Doom,
OpenBSD, “hack – Exploring The Dungeons of Doom,” 2025, OpenBSD Manual Pages, available online in October 2025: https://man.openbsd. org/hack
2025
-
[19]
A Time Series Analysis of Assertions in the Linux Kernel,
J. Ruohonen, “A Time Series Analysis of Assertions in the Linux Kernel,” inProceedings of the 37th International Conference on Testing Software and Systems (ICTSS 2025). Limassol: Springer, 2026, pp. 3–15
2025
-
[20]
Pledge, and Unveil, in OpenBSD,
B. Beck, “Pledge, and Unveil, in OpenBSD,” 2018, BSDCan, Ot- tawa, available online in October 2025: https://www.openbsd.org/papers/ BeckPledgeUnveilBSDCan2018.pdf
2018
-
[21]
Privilege Separation and Pledge,
T. de Raadt, “Privilege Separation and Pledge,” 2016, dotSecurity, Paris, available online in October 2025: https://www.openbsd.org/papers/ dot2016.pdf
2016
-
[22]
veriexec – file integrity subsystem,
NetBSD, “veriexec – file integrity subsystem,” 2017, NetBSD Manual Pages, available online in October 2025: https://man.netbsd.org/veriexec. 8
2017
-
[23]
An Exploratory Study on the Engineering of Security Features,
K. Hermann, S. Peldszus, J.-P. Stegh ¨ofer, and T. Berger, “An Exploratory Study on the Engineering of Security Features,” inProceedings of the IEEE/ACM 47th International Conference on Software Engineering (ICSE 2025). IEEE, 2025, pp. 2470–2482
2025
-
[24]
Infiltrating Security Into Development: Exploring the World’s Largest Software Security Study,
C. Weir, S. Migues, M. Ware, and L. Williams, “Infiltrating Security Into Development: Exploring the World’s Largest Software Security Study,” inProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software E...
2021
-
[25]
Security in Agile Software Development: A Practitioner Survey,
K. Rindell, J. Ruohonen, J. Holvitie, S. Hyrynsalmi, and V . Lepp ¨anen, “Security in Agile Software Development: A Practitioner Survey,” Information and Software Technology, vol. 131, p. 106488, 2021
2021
-
[26]
Review of Existing Analysis Tools for SELinux Security Policies: Challenges and a Proposed Solution,
A. Eaman, B. Sistany, and A. Felty, “Review of Existing Analysis Tools for SELinux Security Policies: Challenges and a Proposed Solution,” inProceedings of the 7th International Conference on E-Technologies: Embracing the Internet of Things (MCETECH 2017). Ottawa: Springer, 20...
2017
-
[27]
An Empirical Study of API Stability and Adoption in the Android Ecosystem,
T. McDonnell, B. Ray, and M. Kim, “An Empirical Study of API Stability and Adoption in the Android Ecosystem,” inProceedings of the IEEE International Conference on Software Maintenance (ICSM 2013). Eindhoven: IEEE, 2013, pp. 70–79
2013
-
[28]
Guidelines for Conducting and Reporting Case Study Research in Software Engineering,
P. Runeson and M. H ¨ost, “Guidelines for Conducting and Reporting Case Study Research in Software Engineering,”Empirical Software Engineering, vol. 14, pp. 131–164, 2009
2009
-
[29]
Software Evolution and Time Series Volatility: An Empirical Exploration,
J. Ruohonen, S. Hyrynsalmi, and V . Lepp ¨anen, “Software Evolution and Time Series Volatility: An Empirical Exploration,” inProceedings of the 14th International Workshop on Principles of Software Evolution (IWPSE 2015). Bergamo: ACM, 2015, pp. 56–65
2015
-
[30]
Empirical Notes on the Interaction Be- tween Continuous Kernel Fuzzing and Development,
J. Ruohonen and K. Rindell, “Empirical Notes on the Interaction Be- tween Continuous Kernel Fuzzing and Development,” inProceedings of the IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW 2019). Berlin: IEEE, 2019, pp. 276–281
2019
-
[31]
Evolution of the Unix System Archi- tecture: An Exploratory Case Study,
D. Spinellis and P. Avgeriou, “Evolution of the Unix System Archi- tecture: An Exploratory Case Study,”IEEE Transactions on Software Engineering, vol. 47, no. 6, pp. 1134–1163, 2021
2021
-
[32]
Dissecting Code Features: An Evolutionary Analysis of Kernel Versus Nonkernel Code in Operating Systems,
Y . Zhao, C. Li, Z. Chen, and Z. Ding, “Dissecting Code Features: An Evolutionary Analysis of Kernel Versus Nonkernel Code in Operating Systems,”Journal of Software: Evolution and Process, vol. 37, no. 1, p. e2752, 2025
2025
-
[33]
Does OpenBSD and Fire- fox’s Security Improve with Time?
J. Shi, D. Zou, S. Xu, X. Deng, and H. Jin, “Does OpenBSD and Fire- fox’s Security Improve with Time?”IEEE Transactions on Dependable and Secure Computing, vol. 20, no. 4, pp. 2781–2793, 2023
2023
-
[34]
pledge – restrict system operations,
OpenBSD, “pledge – restrict system operations,” 2025, OpenBSD Man- ual Pages, available online in October 2025: https://man.openbsd.org/ pledge.2
2025
-
[35]
unveil – unveil parts of a restricted filesystem view,
——, “unveil – unveil parts of a restricted filesystem view,” 2025, OpenBSD Manual Pages, available online in October 2025: https: //man.openbsd.org/unveil.2
2025
-
[36]
A Dataset for a Paper Entitled “A Measurement Study on the Adoption of Pledges and Unveils in the OpenBSD Operating System
J. Ruohonen, K. Sierszecki, and A. Tiwari, “A Dataset for a Paper Entitled “A Measurement Study on the Adoption of Pledges and Unveils in the OpenBSD Operating System”,” 2025, Zenodo, available online: https://doi.org/10.5281/zenodo.18006561
2025 doi
-
[37]
Ghosts of Unix Past: A Historical Search for Design Patterns,
N. Brown, “Ghosts of Unix Past: A Historical Search for Design Patterns,” 2010, Linux Weekly News (LWN), available online in October 2025: https://lwn.net/Articles/411845/
2010
-
[38]
Analyzing a Decade of Linux System Calls,
M. Bagherzadeh, N. Kahani, C.-P. Bezemer, A. E. Hassan, J. Dingel, and J. R. Cordy, “Analyzing a Decade of Linux System Calls,”Empirical Software Engineering, vol. 23, pp. 1519–1551, 2018
2018
-
[39]
Characterizing and Identifying Reverted Commits,
M. Yan, X. Xia, D. Lo, A. E. Hassan, and S. Li, “Characterizing and Identifying Reverted Commits,”Empirical Software Engineering, vol. 24, pp. 2171–2208, 2019
2019
-
[40]
DASP: A Framework for Driving the Adoption of Software Security Practices,
E. Larios-Vargas, O. Elazhary, S. Yousefi, D. Lowlind, M. L. W. Vliek, and M.-A. Storey, “DASP: A Framework for Driving the Adoption of Software Security Practices,”IEEE Transactions on Software Engineer- ing, vol. 49, no. 4, pp. 2892–2919, 2023
2023
-
[41]
Vulnerability Counts: A Rejoinder,
Communications Staff, “Vulnerability Counts: A Rejoinder,”Communi- cations of the ACM, vol. 68, no. 12, p. 7, 2025
2025
-
[42]
Stop Using Vulnerability Counts to Measure Software Security,
A. Meneely and B. Keller, “Stop Using Vulnerability Counts to Measure Software Security,”Communications of the ACM, vol. 68, no. 9, pp. 34–36, 2025
2025
Reviewed July 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.