{"id":"9b928f67-83b9-43e3-902e-64531e66fa6e","arxiv_id":"2607.09689","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":1,"one_line_summary":"A fork-aware MapReduce contract that pools independent Gaussian estimates by inverse-information weighting while carrying evidence IDs and lineage, rejecting literal duplicate evidence.","lead":"This paper introduces a record format and reducer so MapReduce-style jobs that fork sandboxes can keep track of the evidence behind each result instead of collapsing outputs to a vote. A reference implementation pools independent Gaussian estimates with inverse-information weighting and blocks exact reuse of evidence identifiers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Duplicate rejection is exact only for declared ID strings; relabeled or inherited shared evidence bypasses the guard, leaving the paper's motivating failure unaddressed.","rationale":"The algebraic core—associative inverse-information pooling and Cochran's Q residual—is correct and carefully scoped. The reader's weakest assumption identifies the independence/common-target scope and trust semantics; my concern focuses on the trust semantics side: the 'exact duplicate-evidence rejection' claim is structurally limited to exact ID strings. The paper explicitly concedes this in Section 2 and Section 5, so there is no hidden internal contradiction. However, this limitation is load-bearing for the central claim's practical value: the motivating repeated-evidence failure is not solved in realistic reuse scenarios unless IDs are authenticated and bound to content. The proposed concrete test would make the gap measurable and would likely reproduce overconfident pooling, but because the paper already flags the limitation, the verdict remains CONDITIONAL rather than moving to REJECT.","tokens_in":6847,"tokens_out":5128,"duration_ms":52361,"concrete_test":"Run a four-worker trace from one 141MB snapshot. Workers A and B each analyze the same 500 ground-truth observations but declare different evidence ID lists (original vs. relabeled column IDs); workers C and D use genuinely independent data. Compare the pooled total precision P and 95% Wald interval width against the same reduction with A and B collapsed into a single worker. If the duplicated run reports roughly twice the precision and a correspondingly narrower interval, the exact-overlap guard does not confer evidence independence, confirming the limitation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's motivating failure is that branches reusing a model, prompt, repository, tests, observations, or execution ancestor can amplify one repeated error into high-confidence consensus. The reducer's guard (Section 2, 'Trust semantics') rejects only nonempty evidence identifier sets that are literally identical; relabeled data, empty IDs, and shared noise fall outside the check, and Section 5 defers any correlation model. Thus the headline benefit—'exact duplicate-evidence rejection'—is exact only for caller-supplied strings, not for evidence. In the common case where two workers are restored from the same snapshot with the same base model and overlapping data but different prompts or relabeled columns, the evidence ID sets differ, and inverse-information pooling multiplies duplicated evidence, narrowing the pooled covariance. The lineage union records the common root but the reducer does not use it. This is a scope limitation, not an algebraic inconsistency; the paper is transparent about it. But it means the central contract's practical value depends on unauthenticated, easily spoofed ID equality, and the motivating consensus-amplification failure survives in exactly the settings the paper names.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an 'evidence-aware' reduction contract for forked compute: each worker emits a record (θhat_k, J_k, n_k, E_k, L_k, m_k), and independent summaries are merged by adding P_k = n_k J_k, q_k = P_k θhat_k, and c_k = θhat_k^T P_k θhat_k. The pooled estimate is θhat = P^{-1} q, the residual Δ = c − q^T P^{-1} q becomes Cochran's Q in the scalar inverse-variance case, and the product integral is evaluated in closed form. Evidence IDs and fork lineage are carried separately, with a guard that rejects repeated nonempty evidence identifiers. The reference implementation validates the algebra, compares information pooling with unweighted averaging on synthetic logistic shards, and exercises the snapshot-to-worker-to-reducer path on several platforms. The authors explicitly scope the statistical claims to independent estimators of a common parameter and defer a correlation/dependence model to future work.","tokens_in":7049,"tokens_out":13871,"duration_ms":144281,"significance":"If the claims hold, the paper provides a clean and honest packaging of classical inverse-information pooling as a MapReduce-style reduction contract. The algebraic core is standard, but the contribution lies in the interface, the provenance fields, the duplicate-ID guard, and a reproducible artifact with unit and synthetic checks. The paper is transparent about its independence and common-target assumptions, and about the fact that turning lineage into a dependence model remains open. The artifact and the explicit separation of execution fan-out from evidence fan-out are useful for system builders. However, the practical protection against the motivating failure—amplified false consensus from shared evidence—is limited to exact declared-ID reuse; relabeled data, shared noise, and partially overlapping evidence are outside the guard, as the paper itself concedes.","major_comments":[{"comment":"The duplicate guard's semantics are ambiguous and potentially inconsistent. The abstract says the implementation 'rejects repeated nonempty evidence identifiers', and Table 2 says 'Repeated nonempty ID raises an error', but §2 says the reducer 'rejects exact overlap by default'. If 'exact overlap' means that the entire evidence-ID sets must be identical, then partial overlap—e.g., E_1={e1,e2}, E_2={e1}—is not rejected, even though e1 is literally reused and the 'repeated identifier' wording is violated. This distinction is load-bearing for the 'exact duplicate-ID rejection' contribution. Please specify whether the guard rejects any nonempty intersection or only set equality, align the abstract/Table 2 with the implemented behavior, and discuss the partial-overlap case explicitly.","section":"§2 Trust semantics; Table 2"},{"comment":"The motivating scenario is that shared models, prompts, repositories, or ancestors can amplify one repeated error into high-confidence consensus. The contract as presented does not prevent that failure except for exact declared-ID reuse; §2 explicitly says relabeled data, empty IDs, and shared noise fall outside the check, and §5 defers any correlation model. Since this limitation is central to the paper's value proposition, the abstract and introduction should state up front that the system does not solve the motivating false-consensus problem, but only records the evidence and lineage needed to address it later. As written, the opening promises more than the contract delivers.","section":"Abstract and §1 vs. §5"}],"minor_comments":[{"comment":"The phrase 'exact overlap' should be replaced with a precise set-theoretic description (e.g., 'nonempty intersection' or 'identical sets') to avoid the ambiguity noted above.","section":"§2, Trust semantics"},{"comment":"The islo row reports 255/256 successes but the single failure is not discussed. Even if it is a platform/capacity artifact, the reader should be told whether it affected the statistical trace or the timing summary.","section":"§4, Table 1"},{"comment":"The phrase 'the integration measure' is vague; specify the Lebesgue measure on R^p. Also note that the closed-form evaluation assumes the Gaussian kernels are unnormalized with unit height as defined in Eq. (6).","section":"§3, Eq. (9)"},{"comment":"The contract carries no estimand/target identifier, so dimension equality does not imply common target. This is acknowledged in the trust-semantics paragraph, but it deserves a more prominent place in Fig. 1 or in the wire-record definition, since the Gaussian pooling is only valid under the common-target assumption.","section":"§2, Wire record"},{"comment":"The statement that information pooling 'improves on unweighted averaging in this check' is adequately hedged, but the choice of a single eight-seed synthetic configuration makes this illustrative rather than a general efficiency claim. A sentence explicitly saying that no broad statistical efficiency conclusion is drawn would help.","section":"§4, Logistic check"}],"recommendation":"major_revision","confidential_remarks":"The algebraic core and artifact are sound, and the paper is refreshingly explicit about its assumptions and limitations. The main concern is the duplicate-guard semantics: if the implementation rejects only identical evidence-ID sets, the central 'exact duplicate-ID rejection' contribution is narrower than the abstract and Table 2 suggest, and the motivating failure mode is only partially addressed. The paper may be publishable after the guard semantics are clarified and the abstract/intro are aligned with the actual scope. The evaluation is thin for a systems paper, but the synthetic checks and the snapshot trace are appropriate for the narrow claims made."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful and unusually honest systems paper, not a statistics paper. The algebraic core — inverse-information pooling, Delta as Cochran's Q — is textbook, and the authors say so. What's new is the worker record: theta_hat, J, n, evidence IDs, lineage, metadata, plus a mergeable numeric state that is associative in any tree order and carries disagreement through the reduction. The reference implementation appears to match the formulas, and the integration trace does exercise a real snapshot-to-worker-to-reducer path. The seeded logistic check, forged-precision check, and the explicit statement that the stress clip is a heuristic without Byzantine guarantees all read as genuine evidence, not decoration.\n\nThe main soft spot is the gap between the motivating failure and the delivered guard. The paper opens with branches reusing a model, prompt, repository, tests, or observations, and warns that counting outputs can manufacture consensus. The reducer's only protection is exact overlap of nonempty evidence ID sets. Relabeled data, empty IDs, shared noise, or two workers restored from the same snapshot with the same base model but different prompts will carry different ID sets, so inverse-information pooling treats duplicated evidence as independent and overstates precision. The paper is transparent about this in Sections 2 and 5 — it says empty identifiers, relabeling, and shared noise fall outside the check, and defers a dependence model to future work. That is a scope limitation, not an internal inconsistency. But it is the load-bearing limitation: the promised 'exact duplicate-evidence rejection' is exact for caller-supplied strings, not for evidence. A production runtime would need authenticated evidence manifests before the guard means much. The lineage field records the shared root but the reducer doesn't use it — which the paper also admits.\n\nA smaller concern: the determinant-cap/median-MAD stress clip is a hand-tuned heuristic; the paper calls it that, so I don't count it against the core. I'd also want the artifact pinned to a commit hash and a one-command reproduction, though the tests and trace are enough to believe the implementation matches the math.\n\nBottom line: the math is sound, the assumptions are stated, and the limitations are in the paper rather than hidden. The central promise is narrower than the abstract's opening suggests, and that should be fixed in revision. For anyone building evaluation or agent pipelines where multiple branches can share a snapshot, this is a useful scaffold and a clear statement of what a real reducer needs. I'd send it to review and would cite it for the contract, with the caveat that the dependence problem remains open.","headline":"A clean, honest systems contract for evidence-aware reduction of independent Gaussian worker summaries; the duplicate-evidence guard is literal-ID-only, so the motivating shared-evidence failure survives in exactly the cases the paper names.","tokens_in":7560,"tokens_out":2178,"would_cite":true,"duration_ms":21661,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small evidence-aware worker record lets forked branches combine as independent Gaussian summaries and rejects duplicated evidence before it can become false consensus","keywords":["evidence-aware reduction","forkable compute","inverse-information pooling","duplicate evidence rejection","Cochran's Q","Gaussian/Wald summaries","provenance lineage","snapshot sandboxes"],"falsifier":"Split a single dataset into two shards, hand each shard to a worker under a different evidence identifier, and pool: the reducer accepts both, reports a small Delta, and returns a narrower interval than an honest single-sample analysis. This observation demonstrates that the exact-overlap guard is the only protection against duplicated evidence and would falsify any stronger claim that the contract detects relabeled or correlated branches.","tokens_in":6683,"feed_emoji":"🧮","tokens_out":12700,"duration_ms":113888,"temperature":0.7,"pith_summary":"Snapshot-backed sandboxes make execution cheaply forkable, but they do not fork independent evidence: branches that share a model, prompt, repository, or observations can repeat one error many times, and counting outputs treats that repetition as consensus. The paper proposes an evidence-aware reduction contract: each worker reports an estimate, an estimated precision, a sample count, evidence identifiers, lineage, and execution metadata, and the reducer combines only the numeric summary using inverse-information pooling while carrying provenance separately. Under its stated assumptions — independent workers, one common parameter, calibrated precisions — the pooled Gaussian/Wald state merges associatively in any tree order, and the residual disagreement is exactly Cochran's Q in the scalar case, appearing also in the product integral of the pooled kernels. A reference implementation rejects exact duplicate evidence IDs and includes one narrow heuristic guard against forged precision, while explicitly leaving correlation through the fork graph as an open problem. If the contract is right, distributed AI evaluation can get an honest confidence interval only when the reduce step knows whether its branches are independent evidence or copies.","feed_headline":"Reducer stops repeated evidence from becoming false consensus","feed_subtitle":"One worker record per branch exposes shared evidence before confidence is pooled.","key_machinery":"The machinery is the total-precision kernel g_k(theta) = exp(-1/2 (theta - theta_hat_k)^T P_k (theta - theta_hat_k)) with P_k = n_k J_k. Summing the exponent quadratics and completing the square turns the product of worker kernels into one pooled Gaussian, and the same sum produces the standalone residual Delta that survives intermediate merges; this is why the scalar state (P, q, c, N) is associative and commutative. Provenance is handled outside the arithmetic: evidence IDs as a set with exact-duplicate rejection, lineage by order-preserving deduplication. The reference implementation uses Cholesky factorization for the pooled center, covariance, and log-determinant, evaluates the product","core_discovery":"On its own terms, the discovery is that the record (estimate, per-observation information, sample count, evidence IDs, lineage, metadata), pooled as P = sum n_k J_k, theta_hat = P^{-1} sum P_k theta_hat_k, Delta = sum (theta_hat_k - theta_hat)^T P_k (theta_hat_k - theta_hat), is a complete contract for combining independent Gaussian/Wald estimates of one common parameter. The numeric state (P, q, c, N) is additive, so flat, streaming, and tree reductions agree in exact arithmetic; Delta is nonnegative and becomes Cochran's Q in the scalar case. The distinctive move separates arithmetic from provenance: evidence IDs are canonicalized as a set and exact nonempty duplicates are rejected, while","pith_inferences":["Because evidence IDs are self-declared and unauthenticated, a production deployment would need authenticated manifests binding each ID to data, model, and prompt versions; otherwise the duplicate guard stops careless reuse but not deliberate relabeling, as the paper itself notes.","The residual Delta could be converted into a between-worker variance component in a random-effects extension, letting the reducer widen intervals by a method-of-moments estimate when heterogeneity is detected.","A natural next interface is calibrated abstention: when lineage records common ancestors the runtime cannot model, it could refuse to return a narrower interval rather than pooling regardless, which the paper names as the open fork-DAG problem.","Reading each worker kernel as a pseudo-likelihood factor, the closed-form product integral hints at a composite-likelihood interpretation in which Delta/2 acts as a conflict penalty; that connection is not developed in the paper."],"forward_implications":["Any tree shape or streaming order of reduction returns the same pooled estimate and residual, so execution planners can place reducers without changing the statistical answer.","Retries and speculative duplicates that reuse the original evidence token are caught as duplicates, so fault tolerance cannot turn into double counting.","Operators get an explicit disagreement diagnostic: a large Delta flags that the independence/common-target assumptions are not satisfied, and in the scalar case it is Cochran's Q.","The contract is deliberately scoped: its guarantees hold only for independent workers estimating one common parameter, with correlated or adaptively selected branches explicitly left to future work.","The recorded lineage is retained in the final result, giving future correlation-aware reducers the data they need to map shared ancestry to covariance."],"fun_headline_variants":["Fork lineage stops one error from becoming false consensus","Evidence-aware MapReduce catches reused data in AI consensus","Pooling that refuses to double-count fork ancestry","Evidence IDs and lineage keep pooled confidence honest","Branching compute: shared evidence can't inflate consensus"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The contract's guarantees rest on workers being independent, targeting one common parameter, and reporting calibrated precisions, with evidence identifiers that are truthful enough that exact duplicate IDs are the only shared evidence; every one of these can fail silently, and relabeled data, shared noise, or inflated precision makes the pooled interval overconfident.","fun_headline_variants_meta":{"raw":{"variants":["Fork lineage stops one error from becoming false consensus","Evidence-aware MapReduce catches reused data in AI consensus","Pooling that refuses to double-count fork ancestry","Evidence IDs and lineage keep pooled confidence honest","Branching compute: shared evidence can't inflate consensus"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000228,"raw_usage":{"total_tokens":1316,"prompt_tokens":753,"completion_tokens":563,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":503}},"tokens_in":497,"tokens_out":563,"duration_ms":6023,"temperature":1.0,"reasoning_tokens":503,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T10:51:52.604153+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Split a single dataset into two shards, hand each shard to a worker under a different evidence identifier, and pool: the reducer accepts both, reports a small Delta, and returns a narrower interval than an honest single-sample analysis. This observation demonstrates that the exact-overlap guard is the only protection against duplicated evidence and would falsify any stronger claim that the contract detects relabeled or correlated branches.","supporting_citations":[],"review_version":2}