Pith.

Artificial Intelligence · arXiv:2609.39717 v2 · submitted 30 Sep 2026

Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents

Serhii Zabolotnii

The paper says

“Benchmarks, audits, and agent protocols describe performance, permissions, and repair, but not how observed evidence should change an agent's authority during a consequential task.”

more of the abstract

We propose a Runtime Assurance Contract (RAC), a policy-level formal schema binding autonomy boundaries, component eligibility, evidence state, transition policy, human-review capacity, and non-compensatory gates. Under RAC, soft metrics may inform routing, whereas a failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop; aggregate performance cannot authorize action. We define the contract, an evidence record, a permission rule, and five invariants. A deterministic failure-injection study in agentic coding evaluates 174 cases, each a distinct mechanism, under a gate conjunction, a score-only rule, and a restricted protocol baseline. At the published example weights and threshold, the score rule admits 56 of 63 block-required injections and all 24 review-required injections; tuned in hindsight, it matches the conjunction case for case. For positive weights, a positive threshold, binary risk signals, and each signal firing alone on some case, the two rules agree on every case if and only if the threshold does not exceed the smallest weight. On 124 further cases written blind by a model of another vendor, the conjunction itself admits 35 of 79 injections, and so does the tuned score rule: a missing detector is invisible to every decision rule. Eighteen hand-authored traces check version-pinned evidence and review transitions. In a prospective synthetic holdout of 24 episodes, RAC and a separately implemented full stateful baseline both match the labels of two blinded LLM judges. These studies test mechanisms on synthetic cases; they establish neither deployed safety nor cross-domain effectiveness.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 1 Oct 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. A clean, honest mechanism paper: the RAC schema and its non-compensatory conjunction hold up formally, all empirical claims are properly scoped, and the one load-bearing gap — gate correctness — is the paper's own acknowledged frontier. This line is wrong

2 major objections · 3 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at Section 7.9; Eq. (4)

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.85):

“It does not show that the set is correct; no independent oracle defines the right gates.”

Pith says: Objection 1 of 2 · major

The central separation claim — that aggregate performance cannot authorize a consequential action — is fail-closed only relative to (i) the applicability map from actions to G_t and (ii) the faithfulness of each gate's {0,1,⊥} evaluation, and neither is tested by the study.

more of objection 1

The 0/140 RAC admission count in Table 4 demonstrates implementability of the gate set posited in Section 3 and Table 2, not its correctness; Section 7.9 concedes that 'no independent oracle defines the right gates.' Because the conjunction fails asymmetrically and silently, a gate wrongly omitted from G_t, or returning 1 where it should return 0 or ⊥, admits a consequential action exactly as a soft metric would, while a too-strict gate blocks a clean action indistinguishably from a correct block. The concrete test this suggests — injecting a deliberately mis-scoped or forced-1 gate into G_t and verifying that the mechanism admits the action — is not run; the paper instead defers to the deployment study sketched at the end of Section 7.9. I credit the authors' explicit disclosure of this boundary; my point is that it bounds the empirical, but not the formal, content of the headline claim.

The paper says, at Section 7.8; Table 4

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.90):

“Limited rerunnability.The shared static decision log permits a count audit, not reconstruction of the private gate and its input fixtures.”

Pith says: Objection 2 of 2 · major

The central failure-injection probe is not independently rerunnable: Section 7.8 states that the private gate implementation and fixture generator 'are not distributed' and that 'that probe cannot be independently rerun from the public supplement.' The headline empirical contrast — RAC 0/140 versus score-only 120/140 admitted injections (Table 4) — therefore rests on a field-limited decision log that supports count audit but not reconstruction.

more of objection 2

This limitation is disclosed, and the counts are internally consistent with Proposition 1, so it reduces the verifiability of the empirical demonstration rather than invalidating it; the formal equivalence result does not depend on the probe. Distributing the fixture generator, or a sufficient description of how the 280 cases were generated, would materially strengthen the reproducibility of the paper's central empirical claim.

3 minor objections

The paper says, at Sections 1, 2, 7.9

Pith says: minor objection 1 of 3

Several passages have missing spaces between words, e.g., 'herea high-riskworkflow' (Section 1), 'G_SUPERVISIONmakes' (Section 2), and '0.000over-blocking' (Section 7.9); these should be fixed.

The paper says, at Figure 2 caption

Pith says: minor objection 2 of 3

The axis label '5·10 −2' is typeset ambiguously; the caption should state the 19-threshold grid explicitly rather than leaving the reader to infer the swept set from the figure.

show 1 more

The paper says, at Section 7.7

Pith says: minor objection 3 of 3

The blinded judges supplied rationales alongside their labels, but only label agreement is reported; reporting rationale-level agreement (or its absence) would strengthen the claim of independent labeling.

What would settle it

Run the deployment-adjacent study the paper itself prescribes in Section 7.9: take a domain with consequential actions and versioned task traces; have two qualified reviewers freeze a codebook and pre-label injected faults; derive the hard gate set and applicability scope from an independent source (deployment risk analysis), not from the paper's Section 3 list; evaluate all arms against paired copies with a single recorded injected fault, recording harmful releases, false refusals, actual review requests, and service delay. If the independently derived gates admit any pre-labeled consequential fault, or block clean actions at rates comparable to a well-tuned score rule, the gate-correctness premise fails and the contract is formal but vacuous; if RAC's verdicts track the expert labels with near-zero harmful releases and near-zero false refusals, the concern is answered.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 1 Oct 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 1 min 2 s · 15,351 tokens read, 8,971 written
  3. Editor wrote the verdict above 20 s · 16,449 tokens read, 2,378 written
  4. Referee accept: 2 major objections, 3 minor 1 min 43 s · 15,704 tokens read, 11,828 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Worth sending to a referee, plain and simple. This is a mechanism paper that knows it is one, and it never pretends otherwise.

What's actually new is the composition, not the parts. Shields, MAPE-K, guardrail models, least-privilege checks, and the compensatory/non-compensatory distinction (Einhorn, Tversky) are all established and cited. What's new is binding them into one tuple with version-pinned evidence records, a non-compensatory conjunction as the permission rule, retry budgets, and human-capacity gates. Equation 4 is the load-bearing formula and it's clean: permission is act plus every applicable gate returning 1, so soft metrics route but never authorize. Proposition 1 is correct under its stated hypotheses — I checked the logic: controls score zero, each metric fires alone in some injected class, so refusing all injections requires tau <= min wi. The corpus arithmetic is internally consistent too: test-rename is the only two-metric class, which is why the vendor configuration catches exactly that 20 of 100.

What the paper does particularly well is candor. Section 7.9 names the circularity directly: the RAC arm is an implementation of the gates the paper posits, so 0/140 demonstrates implementability, not correctness, and no independent oracle defines the right gates. It flags the hindsight threshold tuning, the repeated templates, the private probe artifacts, the instructional-only blinding, the unavailable model snapshots. It even frames its own needed next experiment — a domain study with pre-labeled injected faults, paired copies, a stateful baseline, and real refusal and delay costs.

The soft spot, in proportion: gate correctness is the load-bearing premise, and the paper concedes it. Equation 4 fails closed only relative to whatever gates a deployer writes, and a conjunction fails asymmetrically and silently — one gate wrongly omitted from the applicable set, or returning 1 where it should return bottom, admits a consequential action exactly as a soft metric would. The evidence record logs the gate outcome, not the correctness of the gate or its applicability. That's a semantic gap, not a formal error, and it's openly acknowledged. The negative result against the vendor config is partly guaranteed arithmetic — the corpus is built so each metric fires alone and tau=0.6 exceeds the smallest weight — but the authors say so themselves, and Proposition 1 is exactly the claim it's stated to be. The holdout is the weakest piece: same model family for judges and corpus authorship, and the full stateful baseline matches RAC, so it shows nothing about architectural superiority. Acknowledged as well.

This is for anyone working on agent governance, audit trails, or post-market monitoring under the EU AI Act. A useful template and a clean statement of why aggregate scores shouldn't authorize consequential actions. Accept it and let it go through review; the design contribution holds.

The strongest objection

Eq. 4 makes permission a conjunction over the applicable subset G_t of G, so the whole mechanism is fail-closed only relative to (i) the applicability map from actions to gates and (ii) the faithfulness of each gate's {0,1,⊥} evaluation. Neither is tested by the study. The RAC arm is an implementation of exactly the gates the paper posits (Section 3, Table 2); its 0/140 refusal rate and the 18/18 transition-trace agreement demonstrate that the specified gates are implementable and that the model matches declared expectations, not that the gate set is correct or complete for any deployed domain. Section 7.9 states this directly ('no independent oracle defines the right gates'), and Section 8 concedes that fail-closed policies can themselves cause harm through delayed care, denied service, and unequal access to reviewers. The unresolved load-bearing step is therefore semantic rather than formal: a conjunction fails asymmetrically and silently. A single gate wrongly omitted from G_t, or returning 1 where it should return ⊥ or 0, admits a consequential action exactly as a soft metric would; a too-strict gate blocks a clean action indistinguishably from a correctly blocking one; and the evidence record (Eq. 3) logs the gate outcome, not the correctness of the gate or of its applicability. Separately, the headline negative result (vendor configuration admits 80/100 block-required and 40/40 review-required injections) is guaranteed arithmetic: the corpus is constructed so that a class fires each metric alone, and τ=0.6 > min_i w_i=0.25 admits every single-fired case. The authors acknowledge this (Sections 7.2, 7.4); it confirms Proposition 1 but does not by itself evidence that conjunctive authorization beats additive authorization under noisy detectors, continuous signals, or correlated metrics, the regime the paper explicitly leaves open.

The reader's weakest-assumption is the right target and the paper concedes it explicitly (Section 7.9 Circularity; abstract; Section 8). I could find no sharper, unacknowledged load-bearing weakness. Proposition 1 is correct under its stated hypotheses (the converse needs every single-fired case refused, giving τ ≤ min_i w_i, and controls score 0), and the corpus arithmetic is internally consistent (review cases fire only the test metric, scoring 0.25 < 0.6, so all 40 are admitted at the vendor point; test-rename alone fires two metrics for 0.70). The transition traces and the LLM-labeled holdout are explicitly scoped as conformance checks and a shared-policy consistency result with no architectural superiority claim; the holdout's review bookkeeping (28 requests vs. 25 completed plus 5 invalid) is reconcilable if invalid events are completed-but-rejected reviews, and in any case does not bear on the central claim. The one genuine soft spot is the semantic one: Eq. 4 is fail-closed only relative to whatever gates a deployer writes, and no evidence binds the Table 2 gate set to any domain's true failure modes. That is exactly the concern the reader flagged, and the reader's ACCEPT is already scoped to a schema-plus-mechanism claim, not deployed safety. Corroborating the reader without adding a new objection, I recommend the verdict stand unchanged, contingent on the paper's own prescribed domain study being the recognized next step.

The referee's report

The formal contribution is sound and precisely stated. Proposition 1 is proved in-line under explicit hypotheses (positive weights, positive threshold, saturated binary signals, zero-signal controls, a solitary-firing case per metric), and the reported counts are internally consistent with the stated weights: test-rename is the only two-metric class (0.45+0.25=0.70), matching the sole 20/100 block-required admissions at τ=0.6, and the clean-region boundary at τ=0.25 equals min_i w_i. The paper ships an executable transition model, hashed public holdout artifacts, a field-limited decision log, and a byte-identical deterministic rerun claim (Section 7.8). Its transparency is a genuine strength: Section 7.9 discloses the circularity (the RAC arm implements exactly the posited gates), the hindsight-best-point selection, the template-reuse dependence, and the non-rerunnability, and Section 8 discloses that fail-closed policies can themselves cause harm. Taken as a design proposition rather than a deployed guarantee — which is the authors' own frame — the contribution is valuable, and the empirical claims are honestly bounded rather than overstated.

Pith's reading of the paper (a machine's summary, not the authors')

This paper argues that an agent's authority to take a consequential action should be decided by a conjunction of mandatory hard gates, not by a weighted risk score. It proposes the Runtime Assurance Contract (RAC), a formal schema in which soft metrics may route work toward retry, switch, escalation, or review, but a failed or unknown mandatory gate always blocks the action; aggregate performance can never waive it. To support the distinction, the paper proves a conditional equivalence: on a constructed binary corpus, a weighted-sum rule reproduces the hard-gate conjunction exactly if and only if its refusal threshold does not exceed the smallest weight, meaning compensation between signals is excluded. It also reports deterministic failure-injection results showing a published score configuration admits 80 of 100 block-required injections while the gate conjunction admits none, plus hand-authored transition traces and a blinded LLM-labeled holdout. The paper is explicit that these synthetic studies establish mechanism, not deployed safety.

What carries the argument. The load-bearing object is the permission rule of Eq. 4: the conjunction over the applicable mandatory hard-gate set G_t = {G_BOUNDS, G_CONTEXT, G_EVIDENCE, G_ELIGIBILITY, G_SUPERVISION, G_RESOLUTION}, each gate returning 1, 0, or ⊥ (unknown), with any 0 or ⊥ blocking the action. It is paired with an evidence record (Eq. 3) that pins each transition to context, rule, and component versions so an approval can be invalidated when those change before act. Proposition 1 is the analytical hinge: it characterizes exactly when a weighted-score rule (Eq. 5) coincides with the conjunction on binary saturated signals, namely if and only if the refusal threshold lies at or below the smallest weight, the precise condition under which no single low-weighted signal can be outweighed by others. The five invariants I1 through I5 — authority separation, fail-closed gates, confidence asymmetry, oversight viability, and evidence continuity — state the contract's normative commitments.

Core claim. The central claim is that authorization is non-compensatory: permission to act is the conjunction Permit = [δ_t = act] ∧ (∧_{g ∈ G_t} g = 1) (Eq. 4), so any gate returning 0 or unknown makes Permit false and no high score elsewhere can offset it, in contrast to score-only authorization (Eq. 5), which refuses once a weighted sum of risk signals crosses a threshold. The proved result, Proposition 1, states that with positive weights, a positive threshold, saturated binary signals, zero-signal controls, and an injected case firing each signal alone, the score rule refuses every injected case and no control case if and only if 0 < τ ≤ min_i w_i. On the reported grid the weighted sum reproduces the conjunction in exactly 30 of 114 configurations, the clean region ends at τ = min_i w_i = 0.25, and the published example threshold of 0.6 lies well outside it, admitting 80 of 100 block-required injections and all 40 review-required ones. The paper further shows that a stateless gate conjunction fails 6 of 18 hand-authored versioned transition traces, and that on a 24-episode, 72-attempt blinded LLM-labeled holdout, RAC and a separately implemented full stateful baseline both match all labels, an observed agreement the paper presents without claiming architectural superiority.

The premise it rests on. The contract's guarantee is only as sound as the premise that the mandatory hard-gate set can be specified correctly and evaluated faithfully in deployment, and the paper's own Section 7.9 concedes that no independent oracle defines the right gates, so a mis-scoped or silently failing gate would fail closed on the wrong condition and make the formal guarantee vacuous.

If the paper is right (Pith's inferences, not the authors')

If the schema is right, any deployed system that authorizes by a weighted sum whose refusal threshold sits above its smallest positive signal weight is, on binary saturated signals, admitting cases a hard-gate conjunction would block; the published example threshold does exactly this.

Aggregate performance, calibration, or confidence measures can route work but cannot, under the contract, authorize a consequential action; that authority rests solely on the gate conjunction and the recorded evidence state.

A failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop, and unavailable or over-capacity human review yields defer or stop, never auto-approval.

Version stability is load-bearing: an approval is invalidated when context, rule, component, input, or output versions change before act, and a missing transition record is an operational failure rather than a later documentation defect.

Hardening the permission interface alone is insufficient: the protocol-only baseline passes every symlink escape and every review-required case, showing that well-formed tool calls can still carry stale, unprovenanced, or insufficiently reviewed task evidence.

The equivalence suggests a cheap audit proxy the paper does not draw: for any score-based authorizer, comparing the configured refusal threshold against the smallest positive signal weight would flag configurations that are structurally permissive on binary signals, a diagnostic deployable before injection testing.

Because the contract's guarantee is only as sound as its weakest gate, its deployment value would hinge on gate-scoping error rates, a quantity the synthetic corpus cannot measure; a testable consequence is that RAC's benefit shrinks as miscoping becomes common.

The zero-signal-control hypothesis doubles as a testability requirement: deployments without clean controls cannot distinguish over-blocking from correct blocking, so the measured zero false-refusal on synthetic controls does not transfer to noisy field signals.

A natural extension left open by the paper is a noisy-signal variant of Proposition 1 with continuous metrics, detectors that miss faults or fire on clean actions; whether the threshold condition degrades gracefully or collapses would decide the contract's practical reach.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Benchmarks, audits, and agent protocols describe performance, permissions, and repair, but not how observed evidence should change an agent's authority during a consequential task. We propose a Runtime Assurance Contract (RAC), a policy-level formal schema binding autonomy boundaries, component eligibility, evidence state, transition policy, human-review capacity, and non-compensatory gates. Under RAC, soft metrics may inform routing, whereas a failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop; aggregate performance cannot authorize action. We define the contract, an evidence record, a permission rule, and five invariants. A deterministic failure-injection study in agentic coding evaluates 174 cases, each a distinct mechanism, under a gate conjunction, a score-only rule, and a restricted protocol baseline. At the published example weights and threshold, the score rule admits 56 of 63 block-required injections and all 24 review-required injections; tuned in hindsight, it matches the conjunction case for case. For positive weights, a positive threshold, binary risk signals, and each signal firing alone on some case, the two rules agree on every case if and only if the threshold does not exceed the smallest weight. On 124 further cases written blind by a model of another vendor, the conjunction itself admits 35 of 79 injections, and so does the tuned score rule: a missing detector is invisible to every decision rule. Eighteen hand-authored traces check version-pinned evidence and review transitions. In a prospective synthetic holdout of 24 episodes, RAC and a separately implemented full stateful baseline both match the labels of two blinded LLM judges. These studies test mechanisms on synthetic cases; they establish neither deployed safety nor cross-domain effectiveness.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-01

The authors' main claim, as the classifier picked it out 0.80

“Under RAC, soft metrics may inform routing, whereas a failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop; aggregate performance cannot authorize action.”

How the main claim is stated: asserted 0.88

What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.98 · human participants 0.11 · controlled comparison 0.94 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.95 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Benchmarks, audits, and agent protocols describe performance, permissions, and repair, but not how observed evidence should change an agent's authority during a consequential task. gap 0.93 0.06 0.06 asserted 1.00
We call this the assurance-transition gap. gap 0.99 0.39 0.04 asserted 0.91
We propose a Runtime Assurance Contract (RAC), a policy-level formal schema binding autonomy boundaries, component eligibility, evidence state, transition policy, human-review capacity, and non-compensatory gates. contribution 0.91 0.84 0.16 asserted 1.00
Under RAC, soft metrics may inform routing, whereas a failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop; aggregate performance cannot authorize action. method 0.59 0.30 0.37 asserted 0.88
We define the contract, an evidence record, a permission rule, and five invariants, and illustrate them in clinical, industrial, and judicial failure probes. contribution 0.78 0.94 0.08 asserted 1.00
We then report a deterministic failure-injection study in agentic coding: 280 constructed cases evaluated by a gate conjunction, a score-only rule, and a restricted protocol baseline. method 0.76 0.95 0.04 asserted 0.99
At the published example weights and threshold, the score rule admits 80 of 100 block-required injections and all 40 review-required injections. result 1.00 0.93 0.09 asserted 1.00
Tuned in hindsight, it matches the conjunction on this corpus. result 0.86 0.90 0.06 asserted 0.99
For positive weights, a positive threshold, binary risk signals, zero-signal controls, and an injected case firing each signal alone, we show that exact agreement holds if and only if the threshold does not exceed the smallest weight. result 0.87 0.90 0.07 asserted 1.00
A separate set of 18 hand-authored traces checks version-pinned evidence and review transitions against simpler policy variants. method 0.88 0.93 0.02 asserted 0.85
In a further prospective synthetic holdout of 24 episodes, two blinded LLM judges assign identical labels to all 72 action attempts; RAC and a separately implemented full stateful baseline both match these labels. result 0.97 0.93 0.08 asserted 1.00
These studies test mechanisms on synthetic cases; they establish neither deployed safety nor cross-domain effectiveness. limitation 1.00 0.53 0.10 asserted 0.98

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.53

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-01: 1 request, 8,920 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (2)
Figure 1 of the paper
Figure 1. RAC state machine. An output does not authorize action directly. Failed or unknown
Figure 2 of the paper
Figure 2. Failure rates of the score-only arm against the refusal threshold, at the most favorable

Figures are the authors', reproduced from the paper.

Cited by 1 Pith paper

Reviewed papers on Pith that cite this one. The full citation record

  1. PROVISO: Provenance-Informed Action Selection for Secure LLM Agents cs.AI · Oct 2026

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment