{"id":"e1a633eb-bda9-46cb-8527-fbe7adbdc6ae","arxiv_id":"2608.01710","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"LLM agents reissuing equivalent actions under fresh tokens cause semantic replay; CapLease shows that durable authorization state, not token representation, prevents duplicate execution.","lead":"The paper identifies 'semantic replay,' where an LLM agent can spend one user authorization multiple times by obtaining a fresh token for each retry, even though every token is single-use. It introduces CapLease, a durable-state authorization layer that prevents reissuance and duplicate effects by binding a confirmation to a canonical action with atomic budget transitions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Residual 0.9% canonicalization evasion in Table 6 violates Proposition 1's 'correct canonicalization' premise, so the unconditional replay-resistance claim is not met by the implementation.","rationale":"The reader's weakest_assumption was that correct semantic canonicalization is load-bearing and that errors break the budget bounds. The paper's Table 6 provides concrete evidence of exactly this error: 0.9% residual evasion. This is the single most load-bearing concern because every bound in Prop. 1 is conditioned on σ being the right equivalence class, and the paper's own data show it is not always. The concern does not invalidate the paper's high-level systems insight—durable authorization state, not token representation, is necessary—but it does mean the implementation's end-to-end replay-resistance claim is not established with certainty. The reader already captured this in a CONDITIONAL verdict, and our specific evidence reinforces it without changing the verdict: the formal conditional result stands, but the practical claim requires better-than-0.9% canonicalization or explicit residual-risk disclosure. Given the paper is a preprint without artifacts, CONDITIONAL remains the appropriate verdict.","tokens_in":10986,"tokens_out":4027,"duration_ms":49209,"concrete_test":"Isolate the evasive equivalent-call pairs from the held-out canonicalization benchmark (Table 6; 324 equivalent pairs, 0.9% evasion). For each, run CapLease with b=1: issue a first call, then submit the evasive variant as a replay. Record N_issue and N_effect. If any pair yields N_issue>1 or N_effect>1, the implementation violates Prop. 1's premise and the claim 'prevents duplicate effects' fails. Additionally, expand the benchmark with adversarial but realistic API variants (unit conversions, default-value omissions, parameter reordering, alias fields) from the 27 high-risk tools to measure whether the residual evasion increases.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central guarantee (Prop. 1, Eq. 4–6) is conditional on 'correct canonicalization': σ(c, γ) must map effect-equivalent calls to the same identity and separate security-relevant differences. But the paper's own canonicalization benchmark (Table 6) reports 'equivalent-call evasion reduced from 13.9% to 0.9%'. That means on a held-out set of 324 equivalent pairs, roughly 3 pairs still produce different σ despite being effect-equivalent. For any such pair, uniqueness over (σ, eu) does NOT deduplicate the authorization instance: the second call gets a fresh record, so N_issue(α, τ) can exceed b, and with a non-idempotent sink N_effect can also exceed b. This is not a hypothetical failure—it is an acknowledged residual error in the component that Prop. 1 assumes to be correct. The paper does not state whether the evasive pairs are security-relevant, nor does it account for this residual in the headline claim that CapLease 'prevents duplicate effects.' The theorem is sound as a conditional statement, but the implementation does not satisfy the condition, so the empirical conclusion is overstated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper defines 'semantic replay' as consuming one token-independent authorization instance beyond its budget through freshly issued token identifiers (replanning, retries, delegation, concurrency, crash recovery). It argues that identifier-local single-use tokens are insufficient and that the issuer must retain durable, monotonic state over the canonical action σ(c,γ), the authenticated confirmation event e_u, and the remaining budget. The proposed CapLease layer stores a durable record indexed by (σ, e_u), enforces atomic Issued→Prepared→Committed transitions, and uses a stable idempotency key k for recovery. Proposition 1 states that, under correct canonicalization, trusted metadata, stable confirmation identifiers, a durable non-rollback linearizable ledger, and authenticated principals, CapLease bounds issuance and admission by b, and also bounds external effects if the sink is idempotent. The evaluation covers 10,152 agent trajectories, 7,896 multi-budget instances, 12,000 fault-injection schedules, a 648-pair canonicalization benchmark, structured-denial recovery, and a matched Server Ledger control.","tokens_in":11215,"tokens_out":5519,"duration_ms":67560,"significance":"If the conditional guarantee is accepted, the paper makes a useful conceptual contribution: it separates authorization identity from token identity and identifies durable state, not token representation, as the key systems requirement for replay resistance in LLM agent tool execution. The proof sketch is not circular and the trust model is explicit; the paper also honestly states that exactly-once external effects require sink idempotency, and the matched Server Ledger is a good control for isolating durable state. The empirical scope is large and the claimed reproducibility artifacts are a strength. However, the central guarantee is conditional on 'correct canonicalization,' and the paper's own canonicalization benchmark reports a 0.9% residual evasion rate, so the unconditional empirical claims are stronger than what is actually demonstrated.","major_comments":[{"comment":"Table 6 reports that tool-specific canonicalization reduces equivalent-call evasion from 13.9% to 0.9%. With 324 equivalent pairs, this leaves roughly 3 effect-equivalent calls that produce different σ. Proposition 1 is explicitly conditional on 'correct canonicalization'; the implementation therefore does not satisfy that premise, and the empirical statement that CapLease 'prevents duplicate effects' is not supported for those residual cases. Because uniqueness over (σ, e_u) is the only mechanism blocking fresh issuance (Eq. 4), an evasive pair can produce a fresh record and violate N_issue(α,τ) ≤ b; with a non-idempotent sink this also breaks N_effect(α,τ) ≤ b. The paper should either show zero security-relevant evasion against a formal equivalence oracle, add a rejection fallback for low-confidence canonicalization, or explicitly qualify all safety claims by the measured residual cano","section":"Realizations and Guarantees / Table 6"},{"comment":"The action identity σ(c,γ) is defined only up to 'tool-specific canonicalization' that must 'map effect-equivalent calls to the same identity.' The paper never formalizes effect equivalence, so 'correct canonicalization' is not a checkable property. This makes Proposition 1's first premise vacuous in the formal statement while the evaluation's canonicalization benchmark uses an apparently different, informal notion. For a load-bearing assumption, the paper should provide a formal equivalence relation, or specify an oracle for the high-risk tool set, and show that the canonicalizer either implements it or rejects ambiguous inputs. Without this, the residual 0.9% cannot be assessed as acceptable or unacceptable.","section":"Problem Formulation and Semantic Replay, Eq. (1)"},{"comment":"For b>1 the paper states that larger budgets 'allocate at most b independently consumable slots' and that the slot counter bounds N_issue, but no atomic slot-allocation transaction is specified. The 7,896-instance multi-budget evaluation (Table 6) exercises this path, yet the protocol description does not show how the counter is incremented atomically with record creation. The proof sketch says 'Uniqueness over (σ,e_u) and the slot counter bound N_issue,' but the slot counter is an implementation detail that is load-bearing for the issuance bound. Please specify the exact conditional update (e.g., a CHECK that the number of allocated slots is < b before insert/update) so that the issuance bound is enforced by the ledger rather than assumed.","section":"Transactional CapLease, multi-budget remark"}],"minor_comments":[{"comment":"Typo: 'Automic' should be 'Atomic' in the CapLease box.","section":"Figure 1"},{"comment":"The affiliation 'Columnbia University' appears to be a typo for 'Columbia University.'","section":"Author affiliations"},{"comment":"The line 'no observed collision' should define what counts as a collision (effect-distinct pairs mapped to the same σ) and should report confidence intervals for the 0.9% evasion rate; with 324 pairs, the binomial uncertainty is nontrivial.","section":"Table 6"},{"comment":"In the proof sketch, N_effect ≤ N_admit ≤ N_issue ≤ b is written with notation inconsistent with the earlier N_issue(α,τ), N_admit(α,τ), N_effect(α,τ). Align the notation.","section":"Realizations and Guarantees"},{"comment":"The supplement is referenced multiple times ('appear in the supplement') but not linked. For a reproducibility-centered paper, include the supplement or an anonymized repository link in the submission.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The manuscript and its reference list contain future-dated arXiv identifiers relative to the current date (e.g., arXiv:2607.06000, arXiv:2608.01710, and the manuscript's own 2608.01710v1 dated 3 Aug 2026). I could not verify these preprints, and this is a serious citation-integrity concern that the editor should check before any acceptance decision. Separately, the paper's central claim is conditional on correct canonicalization, but the reported 0.9% evasion rate means the empirical headline is overbroad; this is fixable in revision by reframing the claims and adding a rejection path, so I recommend major revision rather than rejection on that technical point."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Semantic replay is a real failure mode and this paper is the first to name and formalize it. The budget bounds (Eqs 4-6) and the conditional proof of Proposition 1 are clear and honest. The design choice — durable state over the authorization instance (σ, e_u) with atomic Issue-Prepare-Commit — is a sensible composition of known primitives, and the matched Server Ledger comparison correctly isolates durable state as the operative requirement. The 39.8% equivalent-reproposal rate over 10,152 trajectories is a solid empirical datapoint for the agent-security community.\n\nWhere it gets softer. The stress-test is essentially right: Table 6 reports a residual 0.9% canonicalization evasion. For those ~3 of 324 equivalent pairs, a second (σ, e_u) record is created, N_issue can exceed b, and with a non-idempotent sink the effect bound fails. The theorem is conditional on 'correct canonicalization,' so it's not a logical flaw, but the abstract's 'prevent duplicate effects' is stronger than what the implementation delivers. The paper should either fix the canonicalizer, make the sink key independent of the record ID, or state plainly that the guarantee holds modulo a small TCB residual.\n\nSecond, there is no code or data. The supplement is mentioned but not shipped with the preprint, so the 10,152 trajectories, 7,896 multi-budget instances, and 12,000 fault injections are unverifiable. For a systems paper this is a serious reproducibility gap.\n\nThird, the all-zero CapLease rows are by design, and that's fine, but it means the evaluation shows the protocol does what it was built to do rather than surviving a real adversary. The canonicalization benchmark and the 282 workflow test are self-constructed. That is acceptable for a first characterization, but it limits how much weight the empirical claims can carry.\n\nOverall, the paper is well-argued and the related work is positioned carefully. The main contribution is the semantic-replay framing and the explicit budget semantics, which is worth having in the literature. I'd send this to peer review. The authors should be asked to release artifacts, address the canonicalization residual head-on, and moderate the abstract's claims to match the TCB boundary. If the artifacts appear and the residual is handled honestly, this becomes a solid contribution to agent authorization.","headline":"A genuinely useful formalization of semantic replay and a sound conditional protocol, but the 0.9% canonicalization evasion leaves a real gap between the theorem and the implementation, and the preprint ships no artifacts.","tokens_in":11726,"tokens_out":5699,"would_cite":true,"duration_ms":66133,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that single-use tokens cannot stop a single user authorization from being executed multiple times under fresh token identifiers, and that replay resistance requires durable, monotonic state over the canonical action, the co","keywords":["semantic replay","authorization consumption","single-use tokens","durable state","LLM agent security","replay resistance","idempotency","CapLease"],"falsifier":"Take one authorization with budget b = 1 and submit two tool calls that produce the same external effect but differ only in argument order, naming, or formatting while keeping the same authenticated confirmation. If the ledger admits both and the sink produces two effects, the central claim fails; a systematic version is a benchmark of effect-equivalent call pairs measuring how often uniqueness over (σ, eu) collapses.","tokens_in":10864,"feed_emoji":"🔁","tokens_out":6618,"duration_ms":69722,"temperature":0.7,"pith_summary":"Tool-using LLM agents replan, retry, delegate, and recover, and each of those behaviors can turn one approved action into multiple executions under freshly issued single-use tokens. The paper names this failure semantic replay: exceeding the budget of the authorization itself, not reusing a token identifier. It argues that identifier-local consumption is inherently insufficient, and that replay-resistant execution requires durable state over the token-independent action identity, the authenticated confirmation, and the remaining execution budget. To demonstrate the requirement, it introduces CapLease, a consumption layer that binds an authorization to a canonical action and enforces atomic Issue–Prepare–Commit transitions; with trusted canonicalization, a linearizable ledger, and an idempotent sink, this bounds issuance, admission, and external effects by the authorized budget. The result matters because LLM agents with tool access can turn model errors into payments, messages, and credential changes, and existing authority checks do not limit how often an approved authorization is consumed.","feed_headline":"Durable state, not single-use tokens, stops agent replays","feed_subtitle":"LLM agents repropose the same action under fresh token IDs; CapLease enforces the original budget.","key_machinery":"The central object is the token-independent action identity σ(c, γ) = (Pu, hc, ha, R), which binds a user principal, canonical operation, complete arguments, and target resource into one equivalence class for effect-equivalent calls; the authorization instance α = (σ, eu, b) adds the authenticated confirmation event and budget. CapLease's machinery is a durable ledger enforcing uniqueness over (σ, eu), combined with linearizable compare-and-swap transitions Issued→Prepared→Committed and a stable idempotency key k = H(ν ∥ j ∥ tool); the same records and transitions run server-side in the matched Server Ledger realization. The identity carries the argument by defining what counts as the same a","core_discovery":"The paper's central discovery is that semantic replay—a single user authorization being issued, admitted, or externally materialized beyond its execution budget under fresh token identifiers—cannot be prevented by any identifier-local single-use scheme. Prevention requires the issuer to retain durable, token-independent state over the action identity σ(c, γ) = (Pu, hc, ha, R), the authenticated confirmation event eu, and the remaining budget b. CapLease realizes this by recording a unique durable record per (σ, eu) and moving each budget slot through atomic Issued→Prepared→Committed transitions with a stable idempotency key; under matched centralized assumptions, a Server Ledger that keeps t","pith_inferences":["The budget-bound formulation suggests a general design rule: model authorization as a stateful, budgeted resource rather than a transferable object, and treat the external sink's idempotency as part of the security contract, not an optional convenience.","The reliance on semantic canonicalization implies that the practical ceiling of this approach is set by the tool-schema author; a testable extension is to build adversarial suites of effect-equivalent but lexically different calls for common tool ecosystems and measure how often canonicalization merges or splits the wrong pairs.","The matched Server Ledger result hints that the authenticated manifest may be unnecessary for replay safety; a promising direction is to investigate whether cheaper authenticators suffice when the ledger itself is the point of control.","The structured-denial recovery results suggest replay safety and recovery can be jointly optimized; one extension is to automatically suggest re-canonicalized proposals when a replay is detected, turning a security rejection into a recovery signal."],"forward_implications":["Any single-use token or grant scheme that tracks only its own identifier will permit fresh reissuance for the same authorization; stopping semantic replay requires durable state keyed by (σ, eu).","Replay-resistant consumption splits into three independent bounds—issuance, admission, and external effects—and the third cannot be guaranteed by the ledger alone: the external sink must honor the stable idempotency key.","Authority checking and durable consumption are complementary: upstream checks decide whether an action is authorized, while CapLease decides how many times that authorization may be used.","Across the replanning, retry, delegation, concurrency, confirmation-replay, and crash-recovery scenarios tested, both stateful designs prevent duplicate admission and duplicate effects while preserving legitimate first actions."],"supporting_citations":[{"why":"Supplies the uncertain-outcome model: a caller cannot determine whether a timed-out remote operation took effect, which is the root of semantic replay.","marker":"(Birrell and Nelson 1984)"},{"why":"Provides the lease mechanism CapLease uses to limit the validity interval of stale authorization rights.","marker":"(Gray and Cheriton 1989)"},{"why":"Provides the transaction concept and the two-phase commit limitation that motivate atomic Issue–Prepare–Commit transitions.","marker":"(Gray 1981)"},{"why":"Supplies linearizability as the correctness condition for the compare-and-swap that bounds concurrent admission.","marker":"(Herlihy and Wing 1990)"},{"why":"Defines SUDP, the single-use grant baseline that blocks old-artifact replay but permits fresh reissuance, which CapLease claims to fix.","marker":"(Yu et al. 2026)"},{"why":"Supplies the principle that authorization must be checked at each access to a protected object, which CapLease applies to consumption.","marker":"(Saltzer and Schroeder 1975)"},{"why":"Provides the AgentDojo tool schemas used to evaluate whether the trusted-canonicalization boundary holds on real contracts.","marker":"(Debenedetti et al. 2024)"}],"fun_headline_variants":["Durable state, not tokens, stops agent replay","Replay-proof agent actions need durable state","CapLease: durable authorization blocks semantic replay","Single-use tokens fail; durable state is the fix","Stop agent replays with durable authorization state"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole guarantee rests on the semantic canonicalization step: effect-equivalent calls must map to the same action identity and security-relevant differences must be kept apart, and if that mapping errs the budget bounds no longer hold.","fun_headline_variants_meta":{"raw":{"variants":["Durable state, not tokens, stops agent replay","Replay-proof agent actions need durable state","CapLease: durable authorization blocks semantic replay","Single-use tokens fail; durable state is the fix","Stop agent replays with durable authorization state"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000152,"raw_usage":{"total_tokens":1043,"prompt_tokens":752,"completion_tokens":291,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":220}},"tokens_in":496,"tokens_out":291,"duration_ms":3932,"temperature":1.0,"reasoning_tokens":220,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T22:22:17.001252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one authorization with budget b = 1 and submit two tool calls that produce the same external effect but differ only in argument order, naming, or formatting while keeping the same authenticated confirmation. If the ledger admits both and the sink produces two effects, the central claim fails; a systematic version is a benchmark of effect-equivalent call pairs measuring how often uniqueness over (σ, eu) collapses.","supporting_citations":[],"review_version":1}