{"id":"af92998b-908a-463c-898e-f9a87b0cfd15","arxiv_id":"2607.03056","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Longitudinal counts over 19 OpenBSD releases show continuous growth in pledge and unveil use, weak size correlation, and heavy but not exclusive reliance on I/O promises.","lead":"OpenBSD has steadily rolled out pledge and unveil sandboxing calls across its core programs and libraries over 19 releases. The data suggest these simple system-call restrictions are easier to adopt in practice than much of the security literature claims.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The reader’s weakest-assumption note is accurate and already anticipated in Sections III-B and VI; it does not rise to a load-bearing threat against the paper’s actual contribution (a clean longitudinal count of adoption inside the OpenBSD base system). Because the claim is carefully hedged (“not necessarily … as troublesome”) and the data release allows independent verification, no adjustment to the ACCEPT verdict is warranted.","tokens_in":11036,"tokens_out":366,"duration_ms":3785,"concrete_test":"Re-run the authors’ released parser on the 7.7 src tree after expanding the match to also capture pledge/unveil invocations that appear inside macros, #ifdef blocks, or multi-line calls; if the revised totals differ from the published figures by more than ~10 % for either system call, re-evaluate whether the linear-growth narrative remains intact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper’s central claim—that continuous, wholesale growth in pledge/unveil call counts across 19 OpenBSD releases indicates adoption of system-call minimization is “not necessarily as troublesome as has often been discussed”—is supported by transparent, reproducible counts and is already caveated by the authors (maintenance effort unmeasured; size proxy crude; parser may miss edge cases). The reader correctly flags the string-match + sub-directory operationalization as the weakest assumption, yet that assumption is not load-bearing for the descriptive claim actually advanced: the observed growth and coverage still stand even if a modest fraction of calls are missed or if some high-count programs later required rework. No internal inconsistency, missing control, or unacknowledged confound undermines the reported measurements or the modest inference drawn from them.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper reports a longitudinal measurement of the adoption of the OpenBSD pledge and unveil system calls across 19 releases (5.9–7.7, 2016–2025). Using string matching on .c files under selected source-tree directories, the authors count invocations and their first (or second) parameters, treat each sub-directory as one program/library, and report (i) substantial pre-release adoption followed by roughly linear growth to >700 pledge and ~300 unveil calls, (ii) concentration in /usr/bin and /usr/sbin, (iii) weak Pearson correlations between call counts and a C-file-count size proxy (0.332 / 0.052), and (iv) frequent use of stdio/rpath-style promises and r-permission unveils while still exercising the broader parameter space. From these descriptive results the authors conclude that system-call minimization and sandboxing need not be as troublesome to adopt as prior literature has suggested.","tokens_in":11205,"tokens_out":888,"duration_ms":7730,"significance":"The work supplies the first multi-release quantitative picture of pledge/unveil adoption inside OpenBSD’s own userland, complementing earlier interview and package-snapshot studies. The publicly released dataset and transparent parsing method make the counts reproducible and falsifiable. Even though the inference about “not troublesome” is modest and caveated, the descriptive facts themselves (wholesale continuous growth, weak size correlation, parameter usage patterns) are useful empirical anchors for the software-evolution and OS-security communities. The paper therefore contributes a solid, if limited, measurement baseline rather than a causal claim.","major_comments":[{"comment":"Section V (and the abstract) draw the inference that adoption “is not necessarily as troublesome as has often been discussed.” The manuscript itself notes (II-C, VI) that maintenance effort, debugging cost, and reverts are unmeasured. The observed growth is consistent with low friction but does not by itself demonstrate it; a single sentence that more tightly bounds the claim to “call-count growth and coverage are compatible with relatively low adoption friction” would keep the conclusion proportionate to the evidence.","section":null},{"comment":"Section III-B operationalizes “program/library” as a source sub-directory and size as the count of .c files. Both choices are acknowledged as imperfect, yet they underwrite the ranking in Fig. 3 and the correlations in Fig. 4. A short sensitivity check (e.g., excluding multi-binary directories such as usr.bin/openssl, or reporting LOC instead of file count for a subset) would strengthen confidence that the weak-correlation result is not an artifact of the proxy.","section":null}],"minor_comments":[{"comment":"Fig. 1 caption and surrounding text call the linear fit a “coarse but sensible heuristic”; the same phrasing should appear in the abstract so readers do not over-interpret the trend line.","section":null},{"comment":"Section III-A: the exclusion of distrib/gnu/regress/sys is reasonable, but a one-sentence justification that these trees are not part of the “core userland” under study would help readers unfamiliar with the OpenBSD layout.","section":null},{"comment":"Section IV-C / Fig. 5: five undocumented promises were added ad hoc; listing them (or noting that they are rare) would improve reproducibility.","section":null},{"comment":"Typographical consistency: “pledge” and “unveil” are sometimes italicized, sometimes not; a uniform convention would improve readability.","section":null},{"comment":"References [3] and [4] are central to the motivation; ensuring the final versions (or arXiv identifiers) are cited once they appear would keep the literature trail current.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is a clean, modest measurement paper that fits a software-engineering / empirical-security venue. The central descriptive claims are sound; the two major comments are easily addressable by tighter wording and a brief sensitivity note. I see no reason for a second full review cycle if the authors respond constructively."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is the first multi-year quantitative look at how OpenBSD itself rolled out pledge and unveil across its core userland. Nineteen releases, public dataset, transparent string-match parsing of the source trees, and clear breakdowns by directory, size proxy, and promise frequency. That is new relative to the interview and package-snapshot papers they cite.\n\nWhat they do well is keep the claims descriptive and proportional. Call counts grow steadily (linear trend offered only as a coarse visual), /usr/bin and /usr/sbin dominate, size (C-file count) barely correlates (0.33 / 0.05), stdio and rpath/r dominate but the full set of promises appears, and the frequencies stay stable. Limitations section already owns the crude size proxy, possible missed macro/conditional calls, and the fact that maintenance effort is unmeasured. Dataset is released; methods are reproducible.\n\nThe soft spot is exactly the one the reader flags: treating raw call counts as evidence that adoption is “not necessarily as troublesome as the literature claims.” High counts show coverage and continuity; they do not show low developer cost or absence of later rework. The authors note this themselves, so the inference is modest rather than load-bearing. No circularity, no invented entities, no math problems—just observational counts.\n\nThis is for people who care about empirical adoption of OS security features or software-evolution metrics. It will not change theory, but it supplies the missing longitudinal baseline. I would send it to referees; the work is honest and reusable. Worth a look if you work in that corner; not a must-read for everyone.","headline":"Clean first longitudinal count of pledge/unveil inside OpenBSD’s own tree; solid data release, modest claims, already-flagged operationalization limits.","tokens_in":11756,"tokens_out":425,"would_cite":true,"duration_ms":10868,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"OpenBSD has steadily adopted pledge and unveil sandboxing across its core userland, showing system-call minimization need not be as hard as the literature claims.","keywords":["operating systems","security engineering","attack surface minimization","system call minimization","sandboxing","software evolution","open source","OpenBSD"],"falsifier":"A re-parse of the same trees that recovers substantially fewer live call sites (e.g., after expanding macros or discarding dead or test-only code) or a developer survey showing that high call counts coexist with frequent pledge/unveil-related debugging pain or reverts would collapse the “not troublesome” claim.","tokens_in":11956,"feed_emoji":"🔒","tokens_out":843,"duration_ms":10660,"temperature":0.7,"pith_summary":"This paper measures how widely and quickly OpenBSD developers added the pledge and unveil system calls to programs and libraries over nineteen releases spanning roughly a decade. The calls restrict what a process may do at runtime and which parts of the filesystem it may touch, shrinking the attack surface. Counts rose steadily even before the calls appeared in official releases, with no strong link to program size and with the full range of fine-grained options actually used. The central claim is that these simple, source-level sandboxing tools have been rolled out wholesale inside OpenBSD itself, contrary to frequent claims that system-call minimization is too painful or complex for developers to adopt at scale.","feed_headline":"OpenBSD rolled out pledge and unveil sandboxing at scale","feed_subtitle":"Nineteen releases show steady growth and full use of fine-grained options, contradicting claims that system-call minimization is too hard.","key_machinery":"Longitudinal counts of pledge and unveil call sites (and their first-parameter promises or permission flags) extracted by simple string matching over .c files in successive OpenBSD source trees, treating each sub-directory as one program or library.","core_discovery":"Across nineteen OpenBSD releases the number of pledge and unveil invocations grew continuously and roughly linearly; many programs already used the calls before they shipped in official releases, especially those installed under /usr/bin and /usr/sbin, while program size (proxied by C-file count) showed little correlation with call counts and the full set of promise and permission flags was exercised rather than only a few coarse options.","pith_inferences":["If the observed wholesale adoption is real, other open-source kernels may gain more traction by offering equally simple, source-embedded primitives rather than powerful but complex filter toolboxes.","The lack of size correlation suggests that maintenance cost, not program scale, is the real barrier; measuring commit reverts or code-churn around pledge/unveil sites would be a direct next test.","POSIX-level standardization of a minimal pledge-like interface could transfer the observed adoption pattern beyond the BSD family."],"forward_implications":["Simple, source-level sandbox primitives can achieve near-universal coverage inside a complete operating-system userland without requiring large-program complexity or size.","Linear growth of such call sites can serve as a coarse but usable heuristic for tracking security-feature adoption over multi-year release histories.","The same measurement approach can be applied to other OS-specific sandbox APIs to test whether adoption difficulty is inherent or design-dependent.","Wholesale use of fine-grained promises and path permissions implies that the design of pledge/unveil is flexible enough for both trivial utilities and larger daemons."],"fun_headline_variants":["OpenBSD pledge and unveil adoption grew linearly over 19 releases","Many OpenBSD programs used pledge and unveil before official release","Fine-grained pledge and unveil options fully exercised in OpenBSD","Program size showed little link to pledge or unveil call counts","Sandboxing via pledge and unveil proved adoptable at scale in OpenBSD"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That counting string occurrences of “pledge(” and “unveil(” inside selected .c files, and treating each sub-directory as a single program, accurately reflects successful, low-friction adoption rather than incomplete, conditional, or high-maintenance uses.","fun_headline_variants_meta":{"raw":{"variants":["OpenBSD pledge and unveil adoption grew linearly over 19 releases","Many OpenBSD programs used pledge and unveil before official release","Fine-grained pledge and unveil options fully exercised in OpenBSD","Program size showed little link to pledge or unveil call counts","Sandboxing via pledge and unveil proved adoptable at scale in OpenBSD"]},"model":"grok-4.5","effort":"low","cost_usd":0.004244,"raw_usage":{"total_tokens":1270,"prompt_tokens":748,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":42440000,"prompt_tokens_details":{"text_tokens":748,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":453,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":748,"tokens_out":69,"duration_ms":3851,"temperature":1.0,"reasoning_tokens":453,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T05:10:26.743040+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"A re-parse of the same trees that recovers substantially fewer live call sites (e.g., after expanding macros or discarding dead or test-only code) or a developer survey showing that high call counts coexist with frequent pledge/unveil-related debugging pain or reverts would collapse the “not troublesome” claim.","supporting_citations":[],"review_version":1}