{"id":"4ee3f769-5fcc-4e5c-8747-926f6a5d03b0","arxiv_id":"2608.00947","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Deterministic pre-write admission with frozen planner declarations matches always-serial reliability on conflict-labeled coding pairs, while selective dynamic admission fails closed on undeclared region-level scope in 46 of 90 runs.","lead":"This paper tests a system, Claim Plane, that stops parallel AI coding agents from overwriting each other's work by checking their declared plans before letting them write. On 30 coding tasks it raised the share of fully successful pairs from 23.3% to 50.0%, but only by running most work in sequence.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Oracle-localized context is the load-bearing external-validity threat: the static-vs-parallel reliability gain is untested without gold-patch location hints.","rationale":"The reader's weakest assumption correctly identifies the oracle-localized context and CooperBench labels as the main threats. I focus on the oracle context because it directly affects the construct validity of the interference mechanism: it supplies the location information that agents in real settings must discover on their own, and that discovery process is a primary source of divergent edits. The internal comparison (parallel vs. static) is not confounded because the same context is used everywhere, so the paper's narrow conclusion about the studied setup is sound. However, generalization to realistic coding tasks is untested, and the paper itself labels this as an external-validity threat (§7). The proposed check is feasible using the released artifacts and would empirically settle whether the oracle context changes the qualitative result. Since the paper transparently discloses this limitation and does not overclaim speedup or generality, I do not think the verdict should change; the reader's ACCEPT stands.","tokens_in":8567,"tokens_out":11494,"duration_ms":134611,"concrete_test":"Re-run a subset (e.g., 12 pairs, 6 conflict/6 clean, all three seeds) with the same frozen declarations and arms, but give coders only the repository snapshot and feature descriptions (no gold-patch locations), using a standard context retriever. Compute the paired static-vs-parallel difference on conflict pairs. If the difference is no longer positive by more than the bootstrap CI, oracle context is responsible for the measured gain; if it remains positive and clean pairs still show no gain, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that pre-write admission protects against integration interference is inferred from the differential improvement of conflict-labeled over clean-labeled pairs. All 360 executions used oracle-localized initial context derived from gold-patch locations (§3.4). This removes the need for each agent to locate edit sites, which is exactly the behavior that creates uncoordinated, overlapping file regions in realistic parallel coding. Both the parallel baseline and the Claim Plane arms used this context, so the internal comparison is fair, but the interference failure mode itself may be altered: with gold hints, agents' edits are anchored to the correct regions, potentially under-representing how often independently chosen regions collide. The paper acknowledges this in §7 as an external-validity threat, but for the central claim it is more load-bearing than a generic limitation: if the gains vanished without oracle hints, the conclusion would not transfer to any real deployment. No evidence is provided that the differential survives a realistic retrieval setup, and the clean-stratum null result (which supports the coordination interpretation) is also contingent on the same context.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports a confirmatory study of Claim Plane, a deterministic pre-write admission mechanism for parallel coding agents. Using 30 frozen CooperBench feature pairs (15 conflict-labeled, 15 clean), three coder seeds, and four coordination arms (parallel, static Claim Plane, dynamic Claim Plane, always serial), it collects 360 executions. The main findings are that static admission recovers always-serial reliability (pair pass 50.0% vs. 23.3% for parallel; integration success 96.7% vs. 65.6%) by serializing 96.7% of executions, while the dynamic policy preserves clean concurrency better but fails closed on undeclared scope in 46/90 executions. The paper concludes that pre-write admission is a viable reliability mechanism but does not provide parallel speedup. The data, hashes, frozen plans, and clustered bootstrap analysis are released for reproduction.","tokens_in":8877,"tokens_out":10219,"duration_ms":110751,"significance":"If the results hold, this is a useful empirical datapoint for multi-agent coding coordination. The study's artifacts, hashing, frozen planner output, and cluster bootstrap are strong reproducibility features, and the paper is unusually candid about the lack of wall-clock speedup and the dynamic policy's operational failure. The seven-cluster structure makes statistical precision limited, but the aggregate pattern is internally consistent and clearly presented. The main value lies in demonstrating that a conservative, declaration-based admission policy can achieve always-serial-level reliability, while also showing the selectivity gap that remains for practical concurrency.","major_comments":[{"comment":"The primary estimand is not defined precisely. The text states a 'paired task-cluster difference' of +26.7 percentage points and justifies it by noting that static and always-serial arms had identical pair-pass rates within every cluster. However, the unweighted mean of per-cluster differences need not equal the aggregate 26.7 points when cluster sizes differ (which they likely do across 30 pairs and 7 tasks). The claim only implies that the static-versus-parallel difference equals the always-serial-versus-parallel difference in each cluster. Please state explicitly whether the quoted difference and the bootstrap CI are computed as (a) the difference in pooled rates, (b) the unweighted mean of cluster-level rate differences, or (c) a cluster-size-weighted mean. The reproducibility of the headline result depends on this.","section":"§3.5 and §4.1"},{"comment":"The oracle-localized initial context is a load-bearing external-validity threat. The abstract and conclusion state that pre-write admission 'can protect parallel coding work from integration interference,' but the entire experiment uses initial context derived from gold-patch locations. This removes the edit-siting problem, which in real settings is a major source of uncoordinated region collisions. The internal comparison is fair because all arms share the same context, but the claim as worded goes beyond the tested conditions. I recommend either adding a non-oracle retrieval-based condition (even on a subset of pairs) or explicitly limiting the conclusion to the oracle-assisted setting. The §7 acknowledgment is appropriate but does not alter the abstract/conclusion wording.","section":"§3.4 and §7"},{"comment":"The statement that 'Static Claim Plane was never worse than parallel execution at the task-cluster level for the primary outcome' is not supported by any per-cluster table. Given only seven clusters, this is a checkable claim and should be shown. If the per-cluster data are omitted for space, add an appendix table with cluster-level pair-pass rates for all four arms. This is a load-bearing part of the dominance interpretation.","section":"§4.1"}],"minor_comments":[{"comment":"Typo: 'da ta' should be 'data'.","section":"§4.4"},{"comment":"The ten-line proximity rule used to classify region undercoverage is ad hoc and not defined in the method. Please state the exact rule and, ideally, provide a sensitivity analysis showing how the 15/45 versus 30/45 split changes with the threshold.","section":"§4.4"},{"comment":"The study is described as 'pre-specified confirmatory,' but no pre-analysis plan or protocol is cited. If one exists, please link it; if not, temper the word 'confirmatory' to 'prospective' or explain the sense in which it is confirmatory.","section":"§2.3/§3.5"},{"comment":"The sentence 'Five of the 15 unique conflict pairs were not serialized under the frozen dynamic declarations' is clear, but consider stating explicitly that this means all three seeds of those pairs were un-serialized, for readers who might otherwise infer partial serialization.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"The study is solid in execution and transparency, but the primary estimator needs a precise definition, and the oracle-context caveat needs to be reflected in the paper's central claims. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection. The released artifacts will help the authors address the estimator question quickly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is the paper that actually does the confirmatory work the earlier design paper promised, and it is honest about what it does not show. Static Claim Plane takes 30 CooperBench pairs and shows a reliable jump in pair pass from 23% to 50%, matching always-serial, with the effect concentrated in conflict-labeled pairs. The dynamic arm fails closed in half the runs, which is reported clearly and becomes a useful failure anatomy rather than a hidden flaw. The 360-execution dataset, frozen plans, and clustered bootstrap are all public, which is exactly the kind of reproducibility the field needs.\n\nWhat's genuinely new: the 360-execution dataset itself, the paired cluster-bootstrap estimate, and the 46-case analysis of dynamic scope failures. The paper does not oversell. It explicitly says it shows reliability gains, not wall-clock speedup, and that the static policy basically collapses to serial execution. That discipline is a real strength. The conflict/clean stratification supports the coordination interpretation because clean pairs show no improvement under static admission.\n\nThe main soft spot is the one the paper acknowledges: oracle-localized initial context from gold-patch locations. The stress-test note worries this is load-bearing. I think the note is partially right. The oracle context makes the internal comparison fair, and it may even inflate the parallel baseline by removing mislocalization errors, so the direction of the bias is not obvious. But it does mean the measured failure mode is 'two agents editing known correct regions that happen to overlap,' not 'agents locating conflicting regions in the first place.' That is a narrower claim than 'pre-write admission protects against realistic parallel interference.' The paper says as much in Section 7. It is a real external-validity constraint, but it is not an internal error, and I would not call it fatal. The clean-stratum null result is also conditional on the same context, so the coordination interpretation should be read as provisional.\n\nThe other soft spots are statistical: seven clusters, wide CIs (9.6 to 60 points). The clustered bootstrap is appropriate, and the paper is transparent about the width. That is a limitation, not a flaw. Also, dynamic failures may partly reflect this specific planner calibration, which the paper acknowledges.\n\nBottom line: this deserves a serious referee. It is precisely the kind of confirmatory study with released artifacts that the agent-coding subfield needs more of. I'd send it out.","headline":"A transparent confirmatory study that delivers the reliability result it promised, with the main caveat being external validity (oracle context), not internal error.","tokens_in":9297,"tokens_out":2624,"would_cite":true,"duration_ms":29745,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 360-execution confirmatory study finds deterministic pre-write admission raised pair pass from 23.3% to 50.0% and integration success from 65.6% to 96.7%, by serializing 96.7% of executions.","keywords":["parallel coding agents","claim-based admission","pre-write coordination","dynamic scope","CooperBench","verification","selective concurrency","change intent"],"falsifier":"Re-run the same 30 frozen pairs under static Claim Plane with the oracle-localized context replaced by unmodified repository state. If the conflict-pair pass rate falls from 60.0% toward the 6.7% parallel baseline, the admission gain is an artifact of gold-patch hints rather than of pre-write admission. A second check: execute the static and always-serial arms with genuinely concurrent provider calls; if their reliability is identical, the mechanism is indistinguishable from serial execution under realistic scheduling.","tokens_in":8473,"feed_emoji":"🤖","tokens_out":9125,"duration_ms":89155,"temperature":0.7,"pith_summary":"This paper asks whether a deterministic gate placed before any repository write can make parallel coding agents reliable. Each agent declares the exact files and operations it intends to touch, and a control plane admits or serializes overlapping declarations before code is written. In a frozen 30-pair, three-seed, 360-execution study, static admission raised pair pass from 23.3% under unconstrained parallel execution to 50.0%, lifted conflict-labeled pairs from 6.7% to 60.0%, and raised integration success from 65.6% to 96.7%, matching the always-serial arm. The catch is that static admission serialized 96.7% of all executions, so most of the reliability gain came from collapsing parallel work into serial work; dynamic admission preserved far more concurrency but failed closed on undeclared scope in 46 of 90 runs. The paper concludes that pre-write admission is a viable reliability mechanism while useful parallel speedup remains an open problem.","feed_headline":"Pre-write admission doubles parallel coding pair pass to 50%","feed_subtitle":"Reliability gain came by serializing 97% of runs, so parallel speedup remains an open problem.","key_machinery":"The central object is the ChangeIntent: a versioned pre-write declaration of an agent's exact base revision, typed resources, dependencies, and operations, with each operation marked committed or contingent. A deterministic control plane uses it as the authority for admission: committed scope participates immediately, contingent scope can be promoted atomically when a mutation needs it, and anything outside both is fail-closed. The static policy commits all declared scope up front and serializes any overlap; the dynamic policy starts with only committed scope and attempts selective concurrency. This machinery carries the argument because it moves coordination from post-hoc merge repair to pr","core_discovery":"The central claim is that deterministic pre-write admission can protect parallel coding from integration interference: if two agents' declared mutation authority overlaps, the control plane serializes them, and if a mutation falls outside declared authority, it is blocked. Static Claim Plane exactly reproduced always-serial reliability, matching the always-serial pair-pass rate of 50.0% at every repository-task cluster and reaching 96.7% integration success, while unconstrained parallel execution passed only 23.3% of pairs. The paper interprets the clean-stratum equality (static and parallel both 40.0% pair pass) as evidence that admission prevented interference rather than improving code qu","pith_inferences":["If the static gain transfers beyond these seven tasks, pre-write admission could be inserted as a model-agnostic safety layer in front of any existing agentic coding runtime, with reliability gains independent of the coder model.","The fact that 45 of 46 blocked mutations targeted files already named in the frozen plan suggests a cheap testable extension: coarse-grained region expansion (e.g., whole-file admission) might close most of the dynamic gap before any learned semantic model is needed.","Because clean-labeled pairs were equally likely to pass under parallel and static execution, the expected value of admission grows with the fraction of overlapping feature pairs in the workload; on mostly independent tasks, a static policy would add overhead without benefit.","The physically sequential API calls mean admission was evaluated on decisions, not realized throughput; measuring true concurrency gains requires a runtime with parallel provider calls, where lease and fencing overhead may change the trade-off."],"forward_implications":["A conservative admission gate can restore always-serial reliability for conflict-prone parallel coding without changing the underlying coder: static Claim Plane matched always-serial at every task cluster on the primary outcome.","The binding constraint in this benchmark is coordination, not model quality: conflict-labeled pairs passed only 6.7% of the time under parallel execution but 60.0% under static admission.","Static admission is not a concurrency technology: it serialized 96.7% of executions, so adopting it trades latency for correctness, and the study claims no physical speedup because provider calls were sequential.","Selective concurrency is possible in principle: dynamic admission serialized only 13.3% of clean-labeled executions (86.7% clean specificity) even though its overall pair pass was low.","The dynamic arm's 46 fail-closed blocks, concentrated in files already present in the declarations, define the next engineering target: bounded scope amendment, followed by better semantic dependency prediction."],"supporting_citations":[{"why":"Defines the Claim Plane architecture under test: versioned ChangeIntents, committed versus contingent scope, admission, promotion, leases, and fencing.","marker":"[1]"},{"why":"Supplies the frozen CooperBench feature pairs, conflict/clean labels, gold-sanity procedure, and combined benchmark checks used as the study population and outcome measures.","marker":"[2]"},{"why":"Names the closest contemporaneous pre-write admission system, against which the paper positions its distinct versioned ChangeIntent mechanism.","marker":"[8]"}],"fun_headline_variants":["Pre-write admission doubles pair pass but serializes 97% of runs","Parallel coding fix: admission control buys reliability by going serial","Admission control lifts pass rate to 50% but kills parallel speedup","Selective concurrency fails to parallelize: 97% of runs serialized","Claim Plane: reliability via serialization, no wall-clock speedup"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that CooperBench's conflict/clean labels and the oracle-localized, gold-patch-derived initial context faithfully represent real parallel-coding interference; if those labels misclassify interference or the oracle context inflates success, the measured gains—especially the 60.0% conflict-pair pass under static admission—will not transfer to realistic settings.","fun_headline_variants_meta":{"raw":{"variants":["Pre-write admission doubles pair pass but serializes 97% of runs","Parallel coding fix: admission control buys reliability by going serial","Admission control lifts pass rate to 50% but kills parallel speedup","Selective concurrency fails to parallelize: 97% of runs serialized","Claim Plane: reliability via serialization, no wall-clock speedup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1402,"prompt_tokens":889,"completion_tokens":513,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":417}},"tokens_in":633,"tokens_out":513,"duration_ms":5504,"temperature":1.0,"reasoning_tokens":417,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:35:27.573516+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same 30 frozen pairs under static Claim Plane with the oracle-localized context replaced by unmodified repository state. If the conflict-pair pass rate falls from 60.0% toward the 6.7% parallel baseline, the admission gain is an artifact of gold-patch hints rather than of pre-write admission. A second check: execute the static and always-serial arms with genuinely concurrent provider calls; if their reliability is identical, the mechanism is indistinguishable from serial execution under realistic scheduling.","supporting_citations":[{"cited_title":"Claim Plane: Enforceable Change Intents and Dynamic Scope for Parallel Coding Agents","cited_arxiv_id":"2607.21909","evidence_quote":"Defines the Claim Plane architecture under test: versioned ChangeIntents, committed versus contingent scope, admission, promotion, leases, and fencing."},{"cited_title":"ATM: CID-Brokered Pre-Write Admission for Multi-Agent Code Co-Synthesis","cited_arxiv_id":"2607.00041","evidence_quote":"Names the closest contemporaneous pre-write admission system, against which the paper positions its distinct versioned ChangeIntent mechanism."}],"review_version":1}