{"id":"dca1fb3a-bcae-4301-ab5e-1f263ccf4c81","arxiv_id":"2609.39717","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A runtime assurance contract schema makes agent authorization fail-closed (failed or unknown mandatory gates block action), and a weighted score rule matches that behavior exactly only when its refusal threshold is no larger than the smallest signal weight.","lead":"This paper proposes a formal 'runtime assurance contract' that ties what an AI agent may do next to hard gate checks, fresh evidence, and available human review, so that no aggregate trust score can override a failed safety check. On synthetic coding-agent cases it shows weighted score rules admit many injected failures unless tuned so that a single signal is enough to refuse, and proves the condition under which score and gates agree.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The load-bearing premise is that the Table 2 gate set can be correctly scoped and faithfully evaluated in deployment; the 0/140 result shows implementability, not correctness, and Section 7.9 concedes no independent oracle defines the right gates.","rationale":"The reader's weakest-assumption is the right target and the paper concedes it explicitly (Section 7.9 Circularity; abstract; Section 8). I could find no sharper, unacknowledged load-bearing weakness. Proposition 1 is correct under its stated hypotheses (the converse needs every single-fired case refused, giving τ ≤ min_i w_i, and controls score 0), and the corpus arithmetic is internally consistent (review cases fire only the test metric, scoring 0.25 < 0.6, so all 40 are admitted at the vendor point; test-rename alone fires two metrics for 0.70). The transition traces and the LLM-labeled holdout are explicitly scoped as conformance checks and a shared-policy consistency result with no architectural superiority claim; the holdout's review bookkeeping (28 requests vs. 25 completed plus 5 invalid) is reconcilable if invalid events are completed-but-rejected reviews, and in any case does not bear on the central claim. The one genuine soft spot is the semantic one: Eq. 4 is fail-closed only relative to whatever gates a deployer writes, and no evidence binds the Table 2 gate set to any domain's true failure modes. That is exactly the concern the reader flagged, and the reader's ACCEPT is already scoped to a schema-plus-mechanism claim, not deployed safety. Corroborating the reader without adding a new objection, I recommend the verdict stand unchanged, contingent on the paper's own prescribed domain study being the recognized next step.","tokens_in":15351,"tokens_out":8971,"duration_ms":61648,"concrete_test":"Run the deployment-adjacent study the paper itself prescribes in Section 7.9: take a domain with consequential actions and versioned task traces; have two qualified reviewers freeze a codebook and pre-label injected faults; derive the hard gate set and applicability scope from an independent source (deployment risk analysis), not from the paper's Section 3 list; evaluate all arms against paired copies with a single recorded injected fault, recording harmful releases, false refusals, actual review requests, and service delay. If the independently derived gates admit any pre-labeled consequential fault, or block clean actions at rates comparable to a well-tuned score rule, the gate-correctness premise fails and the contract is formal but vacuous; if RAC's verdicts track the expert labels with near-zero harmful releases and near-zero false refusals, the concern is answered.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 4 makes permission a conjunction over the applicable subset G_t of G, so the whole mechanism is fail-closed only relative to (i) the applicability map from actions to gates and (ii) the faithfulness of each gate's {0,1,⊥} evaluation. Neither is tested by the study. The RAC arm is an implementation of exactly the gates the paper posits (Section 3, Table 2); its 0/140 refusal rate and the 18/18 transition-trace agreement demonstrate that the specified gates are implementable and that the model matches declared expectations, not that the gate set is correct or complete for any deployed domain. Section 7.9 states this directly ('no independent oracle defines the right gates'), and Section 8 concedes that fail-closed policies can themselves cause harm through delayed care, denied service, and unequal access to reviewers. The unresolved load-bearing step is therefore semantic rather than formal: a conjunction fails asymmetrically and silently. A single gate wrongly omitted from G_t, or returning 1 where it should return ⊥ or 0, admits a consequential action exactly as a soft metric would; a too-strict gate blocks a clean action indistinguishably from a correctly blocking one; and the evidence record (Eq. 3) logs the gate outcome, not the correctness of the gate or of its applicability. Separately, the headline negative result (vendor configuration admits 80/100 block-required and 40/40 review-required injections) is guaranteed arithmetic: the corpus is constructed so that a class fires each metric alone, and τ=0.6 > min_i w_i=0.25 admits every single-fired case. The authors acknowledge this (Sections 7.2, 7.4); it confirms Proposition 1 but does not by itself evidence that conjunctive authorization beats additive authorization under noisy detectors, continuous signals, or correlated metrics, the regime the paper explicitly leaves open.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes Runtime Assurance Contracts (RAC), a policy-level formal schema that separates soft routing metrics from non-compensatory hard gates in authorizing consequential agent actions. It defines the contract tuple C_RA = <B, K, Π, H, E, G> (Eq. 1), a version-pinned evidence record e_t (Eq. 3), a permission rule Permit = [δ_t = act] ∧ ⋀_{g∈G_t}[g(...)=1] (Eq. 4), five invariants (I1–I5), and a minimal gate set (Table 2). The evaluation comprises a deterministic 280-case failure-injection study in agentic coding comparing the gate conjunction with a score-only rule and a protocol-only baseline (Tables 4–5), 18 hand-authored versioned transition traces (Section 7.6), and a prospective synthetic holdout of 24 episodes with 72 action attempts labeled by two blinded LLM judges (Section 7.7). Proposition 1 establishes that, on the binary saturated corpus, the score rule matches the conjunction (refusing all injections and no controls) if and only if 0 < τ ≤ min_i w_i. The paper repeatedly and explicitly scopes its results as mechanism studies that establish neither deployed safety nor cross-domain effectiveness.","tokens_in":15704,"tokens_out":11828,"duration_ms":102617,"significance":"The formal contribution is sound and precisely stated. Proposition 1 is proved in-line under explicit hypotheses (positive weights, positive threshold, saturated binary signals, zero-signal controls, a solitary-firing case per metric), and the reported counts are internally consistent with the stated weights: test-rename is the only two-metric class (0.45+0.25=0.70), matching the sole 20/100 block-required admissions at τ=0.6, and the clean-region boundary at τ=0.25 equals min_i w_i. The paper ships an executable transition model, hashed public holdout artifacts, a field-limited decision log, and a byte-identical deterministic rerun claim (Section 7.8). Its transparency is a genuine strength: Section 7.9 discloses the circularity (the RAC arm implements exactly the posited gates), the hindsight-best-point selection, the template-reuse dependence, and the non-rerunnability, and Section 8 discloses that fail-closed policies can themselves cause harm. Taken as a design proposition rather than a deployed guarantee — which is the authors' own frame — the contribution is valuable, and the empirical claims are honestly bounded rather than overstated.","major_comments":[{"comment":"The central separation claim — that aggregate performance cannot authorize a consequential action — is fail-closed only relative to (i) the applicability map from actions to G_t and (ii) the faithfulness of each gate's {0,1,⊥} evaluation, and neither is tested by the study. The 0/140 RAC admission count in Table 4 demonstrates implementability of the gate set posited in Section 3 and Table 2, not its correctness; Section 7.9 concedes that 'no independent oracle defines the right gates.' Because the conjunction fails asymmetrically and silently, a gate wrongly omitted from G_t, or returning 1 where it should return 0 or ⊥, admits a consequential action exactly as a soft metric would, while a too-strict gate blocks a clean action indistinguishably from a correct block. The concrete test this suggests — injecting a deliberately mis-scoped or forced-1 gate into G_t and verifying that the mechanism admits the action — is not run; the paper instead defers to the deployment study sketched at the end of Section 7.9. I credit the authors' explicit disclosure of this boundary; my point is that it bounds the empirical, but not the formal, content of the headline claim.","section":"Section 7.9; Eq. (4)","authors_acknowledge":"It does not show that the set is correct; no independent oracle defines the right gates.","authors_acknowledge_p":0.85},{"comment":"The central failure-injection probe is not independently rerunnable: Section 7.8 states that the private gate implementation and fixture generator 'are not distributed' and that 'that probe cannot be independently rerun from the public supplement.' The headline empirical contrast — RAC 0/140 versus score-only 120/140 admitted injections (Table 4) — therefore rests on a field-limited decision log that supports count audit but not reconstruction. This limitation is disclosed, and the counts are internally consistent with Proposition 1, so it reduces the verifiability of the empirical demonstration rather than invalidating it; the formal equivalence result does not depend on the probe. Distributing the fixture generator, or a sufficient description of how the 280 cases were generated, would materially strengthen the reproducibility of the paper's central empirical claim.","section":"Section 7.8; Table 4","authors_acknowledge":"Limited rerunnability.The shared static decision log permits a count audit, not reconstruction of the private gate and its input fixtures.","authors_acknowledge_p":0.9}],"minor_comments":[{"comment":"Several passages have missing spaces between words, e.g., 'herea high-riskworkflow' (Section 1), 'G_SUPERVISIONmakes' (Section 2), and '0.000over-blocking' (Section 7.9); these should be fixed.","section":"Sections 1, 2, 7.9","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The axis label '5·10 −2' is typeset ambiguously; the caption should state the 19-threshold grid explicitly rather than leaving the reader to infer the swept set from the figure.","section":"Figure 2 caption","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The blinded judges supplied rationales alongside their labels, but only label agreement is reported; reporting rationale-level agreement (or its absence) would strengthen the claim of independent labeling.","section":"Section 7.7","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"accept"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Worth sending to a referee, plain and simple. This is a mechanism paper that knows it is one, and it never pretends otherwise.\n\nWhat's actually new is the composition, not the parts. Shields, MAPE-K, guardrail models, least-privilege checks, and the compensatory/non-compensatory distinction (Einhorn, Tversky) are all established and cited. What's new is binding them into one tuple with version-pinned evidence records, a non-compensatory conjunction as the permission rule, retry budgets, and human-capacity gates. Equation 4 is the load-bearing formula and it's clean: permission is act plus every applicable gate returning 1, so soft metrics route but never authorize. Proposition 1 is correct under its stated hypotheses — I checked the logic: controls score zero, each metric fires alone in some injected class, so refusing all injections requires tau <= min w_i. The corpus arithmetic is internally consistent too: test-rename is the only two-metric class, which is why the vendor configuration catches exactly that 20 of 100.\n\nWhat the paper does particularly well is candor. Section 7.9 names the circularity directly: the RAC arm is an implementation of the gates the paper posits, so 0/140 demonstrates implementability, not correctness, and no independent oracle defines the right gates. It flags the hindsight threshold tuning, the repeated templates, the private probe artifacts, the instructional-only blinding, the unavailable model snapshots. It even frames its own needed next experiment — a domain study with pre-labeled injected faults, paired copies, a stateful baseline, and real refusal and delay costs.\n\nThe soft spot, in proportion: gate correctness is the load-bearing premise, and the paper concedes it. Equation 4 fails closed only relative to whatever gates a deployer writes, and a conjunction fails asymmetrically and silently — one gate wrongly omitted from the applicable set, or returning 1 where it should return bottom, admits a consequential action exactly as a soft metric would. The evidence record logs the gate outcome, not the correctness of the gate or its applicability. That's a semantic gap, not a formal error, and it's openly acknowledged. The negative result against the vendor config is partly guaranteed arithmetic — the corpus is built so each metric fires alone and tau=0.6 exceeds the smallest weight — but the authors say so themselves, and Proposition 1 is exactly the claim it's stated to be. The holdout is the weakest piece: same model family for judges and corpus authorship, and the full stateful baseline matches RAC, so it shows nothing about architectural superiority. Acknowledged as well.\n\nThis is for anyone working on agent governance, audit trails, or post-market monitoring under the EU AI Act. A useful template and a clean statement of why aggregate scores shouldn't authorize consequential actions. Accept it and let it go through review; the design contribution holds.","headline":"A clean, honest mechanism paper: the RAC schema and its non-compensatory conjunction hold up formally, all empirical claims are properly scoped, and the one load-bearing gap — gate correctness — is the paper's own acknowledged frontier.","tokens_in":16449,"tokens_out":2378,"duration_ms":19947},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"This paper argues that an agent should be authorized to act by a conjunction of mandatory hard gates, not by a weighted risk score, and proves when the two decision rules coincide.","keywords":["runtime assurance contract","non-compensatory gates","fail-closed authorization","authority transitions","agent safety","human oversight","evidence record","weighted-score equivalence"],"falsifier":"A deployment study of the kind the paper sketches in Section 7.9 — consequential actions in versioned task traces labeled before injection by two qualified reviewers under a frozen codebook — in which the specified gate set either releases an action that harms or blocks a legitimate action a competent human reviewer would authorize would show the gates are mis-scoped and the contract's guarantee vacuous; equivalently, any corpus satisfying Proposition 1's hypotheses with τ above min_i w_i that a score rule still fully refuses would falsify the equivalence.","tokens_in":16679,"feed_emoji":"🛡️","tokens_out":6805,"duration_ms":69847,"temperature":0.7,"pith_summary":"This paper argues that an agent's authority to take a consequential action should be decided by a conjunction of mandatory hard gates, not by a weighted risk score. It proposes the Runtime Assurance Contract (RAC), a formal schema in which soft metrics may route work toward retry, switch, escalation, or review, but a failed or unknown mandatory gate always blocks the action; aggregate performance can never waive it. To support the distinction, the paper proves a conditional equivalence: on a constructed binary corpus, a weighted-sum rule reproduces the hard-gate conjunction exactly if and only if its refusal threshold does not exceed the smallest weight, meaning compensation between signals is excluded. It also reports deterministic failure-injection results showing a published score configuration admits 80 of 100 block-required injections while the gate conjunction admits none, plus hand-authored transition traces and a blinded LLM-labeled holdout. The paper is explicit that these synthetic studies establish mechanism, not deployed safety.","feed_headline":"Permission is a gate conjunction, not a score","feed_subtitle":"A weighted sum matches fail-closed gates only when its threshold stays below the smallest risk weight.","key_machinery":"The load-bearing object is the permission rule of Eq. 4: the conjunction over the applicable mandatory hard-gate set G_t = {G_BOUNDS, G_CONTEXT, G_EVIDENCE, G_ELIGIBILITY, G_SUPERVISION, G_RESOLUTION}, each gate returning 1, 0, or ⊥ (unknown), with any 0 or ⊥ blocking the action. It is paired with an evidence record (Eq. 3) that pins each transition to context, rule, and component versions so an approval can be invalidated when those change before act. Proposition 1 is the analytical hinge: it characterizes exactly when a weighted-score rule (Eq. 5) coincides with the conjunction on binary saturated signals, namely if and only if the refusal threshold lies at or below the smallest weight, the precise condition under which no single low-weighted signal can be outweighed by others. The five invariants I1 through I5 — authority separation, fail-closed gates, confidence asymmetry, oversight viability, and evidence continuity — state the contract's normative commitments.","core_discovery":"The central claim is that authorization is non-compensatory: permission to act is the conjunction Permit = [δ_t = act] ∧ (∧_{g ∈ G_t} g = 1) (Eq. 4), so any gate returning 0 or unknown makes Permit false and no high score elsewhere can offset it, in contrast to score-only authorization (Eq. 5), which refuses once a weighted sum of risk signals crosses a threshold. The proved result, Proposition 1, states that with positive weights, a positive threshold, saturated binary signals, zero-signal controls, and an injected case firing each signal alone, the score rule refuses every injected case and no control case if and only if 0 < τ ≤ min_i w_i. On the reported grid the weighted sum reproduces the conjunction in exactly 30 of 114 configurations, the clean region ends at τ = min_i w_i = 0.25, and the published example threshold of 0.6 lies well outside it, admitting 80 of 100 block-required injections and all 40 review-required ones. The paper further shows that a stateless gate conjunction fails 6 of 18 hand-authored versioned transition traces, and that on a 24-episode, 72-attempt blinded LLM-labeled holdout, RAC and a separately implemented full stateful baseline both match all labels, an observed agreement the paper presents without claiming architectural superiority.","pith_inferences":["The equivalence suggests a cheap audit proxy the paper does not draw: for any score-based authorizer, comparing the configured refusal threshold against the smallest positive signal weight would flag configurations that are structurally permissive on binary signals, a diagnostic deployable before injection testing.","Because the contract's guarantee is only as sound as its weakest gate, its deployment value would hinge on gate-scoping error rates, a quantity the synthetic corpus cannot measure; a testable consequence is that RAC's benefit shrinks as miscoping becomes common.","The zero-signal-control hypothesis doubles as a testability requirement: deployments without clean controls cannot distinguish over-blocking from correct blocking, so the measured zero false-refusal on synthetic controls does not transfer to noisy field signals.","A natural extension left open by the paper is a noisy-signal variant of Proposition 1 with continuous metrics, detectors that miss faults or fire on clean actions; whether the threshold condition degrades gracefully or collapses would decide the contract's practical reach."],"forward_implications":["If the schema is right, any deployed system that authorizes by a weighted sum whose refusal threshold sits above its smallest positive signal weight is, on binary saturated signals, admitting cases a hard-gate conjunction would block; the published example threshold does exactly this.","Aggregate performance, calibration, or confidence measures can route work but cannot, under the contract, authorize a consequential action; that authority rests solely on the gate conjunction and the recorded evidence state.","A failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop, and unavailable or over-capacity human review yields defer or stop, never auto-approval.","Version stability is load-bearing: an approval is invalidated when context, rule, component, input, or output versions change before act, and a missing transition record is an operational failure rather than a later documentation defect.","Hardening the permission interface alone is insufficient: the protocol-only baseline passes every symlink escape and every review-required case, showing that well-formed tool calls can still carry stale, unprovenanced, or insufficiently reviewed task evidence."],"supporting_citations":[{"why":"The author's earlier framework treating trust as measure-oriented evidence, whose layer structure the RAC tuple extends from measurement into explicit authority transitions.","marker":"[42]"},{"why":"Vendor documentation supplying the published weighted-sum example configuration, the score-only baseline evaluated in Section 7.","marker":"[36]"},{"why":"The reported agent-harness incident of a bypass persisting across episodes, which motivates the need for a recorded and independently validated authority transition.","marker":"[19]"},{"why":"The decision-research source for the additive-versus-conjunctive distinction that grounds the non-compensatory authorization claim.","marker":"[10]"},{"why":"The empirical study concluding authorization is not a conservative-versus-permissive scalar calibration, supporting the paper's claim that authority is not a dial.","marker":"[41]"},{"why":"The deterministic permission-checking approach that defines the protocol-only baseline evaluated in Section 7.","marker":"[34]"},{"why":"Documentation of the branch-protection options the prospective holdout workflow models for reviews, stale-approval dismissal, and reviewer separation.","marker":"[14]"}],"fun_headline_variants":["Authorization is a conjunction, not a score","Risk scores can't override a failed gate","Trust expires mid-task unless gates stay closed","A gate fails, the score doesn't matter","When thresholds fall below the smallest weight"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The contract's guarantee is only as sound as the premise that the mandatory hard-gate set can be specified correctly and evaluated faithfully in deployment, and the paper's own Section 7.9 concedes that no independent oracle defines the right gates, so a mis-scoped or silently failing gate would fail closed on the wrong condition and make the formal guarantee vacuous.","fun_headline_variants_meta":{"raw":{"variants":["Authorization is a conjunction, not a score","Risk scores can't override a failed gate","Trust expires mid-task unless gates stay closed","A gate fails, the score doesn't matter","When thresholds fall below the smallest weight"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000168,"raw_usage":{"cost":0.000167748,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000167748,"upstream_inference_prompt_cost":0.000133028,"upstream_inference_completions_cost":3.472e-05},"total_tokens":1279,"prompt_tokens":1155,"completion_tokens":124,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":57}},"tokens_in":1155,"tokens_out":124,"cost_source":"openrouter_usage","duration_ms":2632,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":57,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-01T12:53:28.174290+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"A deployment study of the kind the paper sketches in Section 7.9 — consequential actions in versioned task traces labeled before injection by two qualified reviewers under a frozen codebook — in which the specified gate set either releases an action that harms or blocks a legitimate action a competent human reviewer would authorize would show the gates are mis-scoped and the contract's guarantee vacuous; equivalently, any corpus satisfying Proposition 1's hypotheses with τ above min_i w_i that a score rule still fully refuses would falsify the equivalence.","supporting_citations":[{"cited_title":"TRACE: A Metrologically-Grounded Engineering Framework for Trustworthy Agentic AI Systems in Operationally Critical Domains","cited_arxiv_id":"2605.03838","evidence_quote":"The author's earlier framework treating trust as measure-oriented evidence, whose layer structure the RAC tuple extends from measurement into explicit authority transitions."},{"cited_title":"System One Documentation: How to Build with TypeSafe, and Confidence","cited_arxiv_id":null,"evidence_quote":"Vendor documentation supplying the published weighted-sum example configuration, the score-only baseline evaluated in Section 7."},{"cited_title":"Prime Agent: A Self-Improving RLM Harness","cited_arxiv_id":"2608.23552","evidence_quote":"The reported agent-harness incident of a bypass persisting across episodes, which motivates the need for a recorded and independently validated authority transition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The decision-research source for the additive-versus-conjunctive distinction that grounds the non-compensatory authorization claim."},{"cited_title":"Do Coding Agents Understand Least-Privilege Authorization?","cited_arxiv_id":"2605.14859","evidence_quote":"The empirical study concluding authorization is not a conservative-versus-permissive scalar calibration, supporting the paper's claim that authority is not a dial."},{"cited_title":"Progent: Securing AI Agents with Privilege Control","cited_arxiv_id":"2504.11703","evidence_quote":"The deterministic permission-checking approach that defines the protocol-only baseline evaluated in Section 7."},{"cited_title":"About protected branches","cited_arxiv_id":null,"evidence_quote":"Documentation of the branch-protection options the prospective holdout workflow models for reviews, stale-approval dismissal, and reviewer separation."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}