{"id":"a80c0146-4103-4b95-84eb-380e2a09fc1b","arxiv_id":"2607.16109","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A quorum of protocol-compliant but semantically mistaken AI validators can certify an invalid transition; EBFT derives threshold conditions that bound this risk with calibrated budgets eδ and uε.","lead":"This paper formalizes a new failure mode for consensus systems with AI validators: agents that follow the protocol exactly but still approve semantically wrong actions. It introduces two budgets—false endorsements and unusable support—and derives quorum-size rules that keep such errors from becoming certified.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Calibration pipeline targets per-task quantiles, not the sup-tail budgets Definition 4.6 requires; central thresholds remain uninstantiated.","rationale":"The paper's formal contribution is a set of conditional inequalities, and those inequalities are not the weak point. The weak point is the bridge from the formal model to any deployment: the budgets eδ/uϵ are the inputs to Theorem 6.4, and the calibration section is the only proposed way to get them. The manuscript itself states that Table 5 is illustrative and 'does not report experimental results,' and Section 10.2 concedes shift/adversarial selection problems. My reading of Section 8.1 is that the proposed estimators are for P[F_m>e] per calibration task, whereas Definitions 4.3/4.6 require control of P[sup_D F_S>e]. These are different objects; finite samples cannot identify the sup tail without enumerating D or imposing strong extreme-value/stationarity assumptions. So the central claim — 'if eδ/uε are correctly calibrated then q interval works' — is not undermined in logic, but the antecedent has no demonstrated satisfiability path. This is exactly the kind of issue that makes the appropriate verdict conditional rather than accept. The reader's verdict already is CONDITIONAL, so I recommend no change. I mark agreement as partial because the reader located the problem in calibration broadly (labels, shift, synthetic numbers); I sharpen it to a statistical target mismatch in the proposed procedure.","tokens_in":30020,"tokens_out":8297,"duration_ms":80040,"concrete_test":"Construct a finite workload D with 999,999 invalid tasks having F=0 and one invalid task with F=5 (N=7, f=1). Draw a calibration sample of size 1000 i.i.d. from D; with rare-task probability 10^-6, the sample misses the rare task with probability ≈0.999. Apply Section 8.1's Clopper-Pearson/bootstrap procedure: the estimated beδ+ will be 0, while the true Definition 4.6 value is eδ=5. Then q=5 satisfies the agreement threshold ⌊(7+1)/2⌋+1=5 but not q>f+eδ=6; on the rare invalid task, a q-certificate forms, violating Theorem 6.1's conclusion. This demonstrates that the proposed calibration estimates per-task quantiles, not the sup-tail budget, and that the gap is not a sample-size artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The threshold arithmetic in §6 is internally consistent: on Eδ∩Uϵ with known eδ,uϵ, qmin≤q≤qmax delivers exactly the stated conditional guarantees. The load-bearing problem is how eδ and uϵ are obtained. Theorem 6.4 rests on Definitions 4.6/4.7, where eδ and uϵ control the tail of the supremum over the entire workload domain D: P[sup_{invalid (x,s)∈D} F_S(x,s)>e]≤δ. Section 8.1 and Appendix B.3/B.4 estimate an upper quantile of per-task counts F_m from a finite labeled sample, via Clopper-Pearson or bootstrap. A per-task quantile does not control the supremum over D; a rare invalid task class with large F can make the true eδ arbitrarily larger than any finite-sample quantile estimate. Thus even a flawless implementation of the proposed calibration yields beδ+ that is not an upper confidence bound for the eδ used in q>f+eδ. The resulting q can be too small, and semantic certificate validity fails exactly off the calibration sample. The same mismatch applies to uϵ. Section 10.2 acknowledges workload shift and Section 8 says a few hundred examples cannot support 10^-6 claims, but neither addresses the estimator-target mismatch: the method estimates a different statistical object, and Table 5 is explicitly synthetic. The central claim is therefore conditional in a way that has not been shown to be satisfiable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Honest Quorum Problem and an Epistemic Byzantine Fault Tolerance (EBFT) model for agentic validators that are protocol-compliant but can make semantically incorrect judgments. It defines confidence-indexed budgets eδ for coherent false endorsement outside the Byzantine set and uε for unusable support, then derives quorum-threshold conditions for semantic certificate validity, agreement, liveness, and feasible integer thresholds (Section 6). The central result, Theorem 6.4, states that if qmin ≤ q ≤ qmax then the q-signature protocol simultaneously prevents invalid certificates on event Eδ, prevents conflicting certificates deterministically, and permits valid certificates on event Uε. Sections 7–8 propose a two-phase calibration/admission methodology, including a synthetic illustrative budget calculation in Table 5. The paper is framed as a positioning/theory contribution rather than an empirical study.","tokens_in":30309,"tokens_out":10736,"duration_ms":100622,"significance":"If the calibration problem were solved, EBFT would be a conceptually valuable extension of BFT: it separates semantic error concentration from Byzantine equivocation, shows that agreement thresholds depend only on the Byzantine bound f, and explicitly models correlated epistemic faults. The lower-bound results in Section 5 are correctly framed as existence/necessity arguments, and the paper is unusually honest about its conditional guarantees and limitations (Section 10). The threshold arithmetic itself is simple and correct, and the manuscript is transparent that Table 5 is synthetic. However, the practical force of the framework rests entirely on whether eδ and uε can be reliably calibrated, and the proposed calibration does not estimate the quantities required by Definitions 4.6–4.7. As a result, the paper currently delivers a coherent conditional theory with an uninstantiated and, as proposed, mismatched calibration methodology.","major_comments":[{"comment":"Definitions 4.6–4.7 define eδ/uε as upper-tail quantiles of the supremum of false-endorsement (resp. unusable) weight over all invalid (resp. valid) (x,s)∈D: P[sup_D F_S > e] ≤ δ. The calibration in §8.1 and Appendix B.3 estimates, per labeled task m, an upper confidence bound for the quantile of the marginal count F_m (Clopper–Pearson/bootstrap). A per-task quantile does not control the supremum over D. Even for independent tasks, if each task has P(F_m>e)≤δ, the maximum over M tasks exceeds e with probability up to Mδ; the binomial 'number of exceedances' procedure in §8.1 targets the count of tasks exceeding e, not the event that any task does. Hence be+δ is not an upper confidence bound for the eδ used in q>f+eδ, and Table 5's synthetic values do not instantiate Theorem 6.4. The caveats in §10.2/10.3 about workload shift and sample size do not address this estimator-target mismatch.","section":"§8.1, Appendix B.3, Definitions 4.6/4.7"},{"comment":"The exact integer interval qmin ≤ q ≤ qmax is correct, but the claim that the unrounded inequalities N>2f+eδ+uε and N>3f+2uε are 'sufficient in the integral-count setting' is false. These conditions are necessary for the real interval max{f+eδ,(N+f)/2} < q ≤ N−f−uε to be nonempty, but they do not guarantee an integer q exists. Counterexample: N=10, f=2, eδ=1.2, uε=1.2 satisfies both inequalities, yet qmin=max(⌊3.2⌋+1,⌊12/2⌋+1)=7 and qmax=⌊10−2−1.2⌋=6. The exact floor/ceiling test should be presented as the sole feasibility criterion; the unrounded conditions should be described as necessary for real feasibility, not sufficient for integer feasibility.","section":"Theorem 6.4 (population inequalities)"}],"minor_comments":[{"comment":"The 'equivalently' formulas using ceilings are valid only when eδ and uε are integral. For real-valued budgets, ceil(f+eδ+1) can differ from floor(f+eδ)+1 (e.g., f=2, eδ=1.2 gives 5 vs 4). Please clarify explicitly which rounding rule applies when empirical upper endpoints are non-integral.","section":"§6.2, Eq. (6.7)–(6.8)"},{"comment":"The homogeneous row reports beδ=4.2 while the largest false-endorsement coalition is 5. Since beδ is a quantile rather than a coalition size, the relationship should be explained to avoid confusion.","section":"§8.1, Table 5"},{"comment":"If D is an infinite domain, the supremum over D may not be measurable without additional assumptions. The paper should state a measurability or finite-support condition, or restrict D to a finite task set.","section":"§4.4, Eqs. (4.6)–(4.7)"},{"comment":"The evaluation section is a methodology description and reports no experimental results. This is acceptable for a theory paper, but the abstract and introduction should make even clearer that the calibration pipeline is not validated on real data; otherwise Table 5 may be mistaken for empirical evidence.","section":"§8, RQ1–RQ4"},{"comment":"The suggestion to use 'Chebyshev or Markov variants for dependent variables' is vague. Markov-type bounds depend on the actual distributional object being bounded; specify what moments or dependence assumptions are needed to make the conservative adjustment concrete.","section":"§10.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is positioned as part of the authors' OpenKedge research program and relies heavily on their own preprints in the references, which may limit its apparent novelty for a general journal. The main technical blocker is the calibration-target mismatch: the proposed statistical procedure does not estimate the sup-tail budgets required by the theorems. This is fixable within scope (e.g., by defining budgets over a finite labeled task set with a union bound, or by proving a proper extreme-value bound), but until then the central guarantee remains conditional in a way that has not been shown to be satisfiable. The rounding error in the population inequalities is secondary but should be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clean conceptual framework, not a validated protocol. The threshold arithmetic is correct as a set of conditional statements, and the semantic fault model is worth keeping. But the budgets the whole thing rests on—eδ and uε—are never actually shown to be obtainable. The paper should be read as a position paper with formal scaffolding.\n\nWhat is new: the epistemic fault class is a real extension of hybrid fault models. A validator can be authenticated, responsive, non-equivocating, protocol-compliant, and still endorse a semantically invalid transition. The paper's key move is to separate coherent false endorsement (safety) from unusable support (liveness), and to tie the safety risk to the upper tail of a common invalid semantic class. That is a better way to think about \"diversity\" than counting model identities. The agreement threshold depending only on f, not on the epistemic budgets, is also a clean observation.\n\nWhat is good: the paper is honest about its own limits. Section 10 explicitly discusses average-case vs uniform guarantees, distribution shift, hidden common-mode failure, label uncertainty, and black-box APIs. It does not overclaim experimental support: Table 5 is labeled synthetic. The formal statements are careful and the hand proofs are easy to check. Self-citations are numerous but mostly to the authors' own closely related framework papers, which is not a problem here.\n\nWhere it is soft: the central theorems are conditional on Eδ and Uε, and those events are defined so the conclusions follow almost by counting. Theorem 6.1 is basically the definition of Eδ plus the Byzantine bound. That is fine as a framework, but the real work is calibrating eδ and uε, and that is where the paper stalls. Definition 4.6 needs a tail bound on the supremum over the whole domain D. Section 8's calibration estimates an upper quantile of per-task counts F_m. A per-task quantile does not control the supremum. A rare invalid task with a large false-endorsement coalition can make the true eδ much larger than any finite-sample estimate. Table 5 is explicitly synthetic, so the paper gives no evidence that its own conditions are satisfiable. The same mismatch affects uε. The lower-bound theorems are mostly existential and close to tautological. No code, data, or machine-checked proofs ship with the paper; the proofs are simple enough that this is a minor point.\n\nWho this is for: people thinking about AI validators, agentic infrastructure, or quorum-based semantic admission. The framework is useful for framing the problem, not for deploying thresholds. I'd send it to peer review if the venue tolerates framework papers, with the expectation that the calibration section needs to either target the sup-tail quantity or be repositioned as a research agenda. As is, I would not cite it as a protocol.","headline":"A genuinely useful fault model for AI validators with correct conditional threshold math—but the calibration pipeline as written does not instantiate the central budgets, so treat it as a framework, not a deployable protocol.","tokens_in":30811,"tokens_out":3939,"would_cite":true,"duration_ms":39014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes the Honest Quorum Problem: protocol-compliant but semantically mistaken validators can form a certificate for an invalid transition, and a single threshold chosen from two epistemic budgets can make that unlikely whil","keywords":["Honest Quorum Problem","Epistemic Byzantine Fault Tolerance","semantic certificate validity","quorum thresholds","Byzantine fault tolerance","agentic validators","correlated model errors","liveness budgets"],"falsifier":"On a labeled workload, calibrate eδ and uε, compute q from Theorem 6.4, then evaluate held-out invalid and valid tasks. If invalid transitions receive certificates noticeably more often than the claimed δ confidence, or valid transitions fail liveness more often than ε, the sufficiency guarantee collapses.","tokens_in":29848,"feed_emoji":"🛡️","tokens_out":5486,"duration_ms":53257,"temperature":0.7,"pith_summary":"Protocol agreement in a Byzantine fault-tolerant system normally assumes that every non-Byzantine participant correctly applies the transition semantics; agentic validators break that assumption because a compliant, non-equivocating model can still endorse a semantically invalid change. The paper defines the Honest Quorum Problem and shows that agreement and termination alone never imply semantic certificate validity. Its solution, Epistemic Byzantine Fault Tolerance (EBFT), adds two confidence-indexed budgets to the Byzantine bound: eδ for coherent false endorsement and uε for unusable support. The central result is that one integer quorum threshold q, chosen between max(f+eδ, (N+f)/2) and N−f−uε, simultaneously keeps invalid candidates from being certified, prevents conflicting certificates, and allows valid candidates to be certified.","feed_headline":"Honest-but-wrong quorums can still certify invalid transitions","feed_subtitle":"A two-budget fault model yields one quorum threshold that blocks invalid certificates and still admits valid transitions.","key_machinery":"The q-signature semantic-certificate protocol and the two confidence events Eδ and Uε. Validators sign structured judgments bound to a canonical evidence digest; a certificate is just q endorsements for the same candidate and class. The safety event Eδ bounds the upper-tail weight of false endorsements outside the Byzantine set by eδ, and the liveness event Uε bounds unusable support by uε. The protocol's power is that a single integer q can satisfy all three requirements when the interval in Theorem 6.4 is nonempty; the quorum-intersection lemma carries the agreement half.","core_discovery":"The paper's central claim is that agreement is orthogonal to semantic validity: a well-formed certificate can be created entirely by authenticated, responsive, non-equivocating, protocol-compliant validators who share a false semantic judgment. EBFT separates the fault model into three parts: the Byzantine bound f, the coherent false-endorsement budget eδ, and the unusable-support budget uε. The threshold theorem then says that, on calibrated confidence events, choosing any integer q with max(⌊f+eδ⌋+1, ⌊(N+f)/2⌋+1) ≤ q ≤ ⌊N−f−uε⌋ simultaneously gives semantic certificate validity, deterministic agreement under non-equivocation, and liveness. The proof is a quorum-intersection argument: inval","pith_inferences":["Because calibration needs ground-truth labels V(x,s), the scheme is strongest exactly where a deterministic verifier already exists; where labels are unavailable, the thresholds are best read as a risk model rather than a guarantee.","The risk-profile and expiry mechanism points toward a testable operational pattern: any model-weight, prompt, retrieval, or tool change should invalidate the current threshold until recalibration, and an online canary could detect silent drift.","For LLM ensembles, the upper-tail view suggests measuring the largest false-endorsement coalition and the concentration of unusable support, since pairwise agreement statistics cannot determine quorum-level safety.","An adaptive adversary who can steer contexts after calibration is outside the base theorem; a natural extension is to require eδ to hold uniformly over adversarially selected instances or to fold influenceable validators into the Byzantine bound."],"forward_implications":["An operator with calibrated eδ and uε can commit to a single quorum threshold that provides semantic certificate validity, agreement, and liveness at once.","Agreement safety is decoupled from semantic error: the 2q−N > f condition depends only on the Byzantine equivocation bound, so epistemic mistakes do not create forks.","Adding models, providers, or prompts changes safety only if the measured upper tail of false endorsement or unusable support drops; nominal diversity has no inherent value.","If false endorsers are conservatively reclassified as Byzantine, the classical 3F+1 bound reapplies on the safety event, at the cost of ignoring liveness.","Without a concentration bound or another semantic grounding mechanism, no count-only certificate rule can guarantee semantic validity."],"fun_headline_variants":["Agreement ≠ safety: honest quorums can certify invalid states","Honest-but-wrong quorums: agreement is not validity","Epistemic faults: when honest quorums certify false transitions","Consensus isn't correctness when validators share wrong beliefs"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that eδ and uε can be conservatively estimated from labeled calibration tasks in the target domain and stay valid under the live workload; if ground-truth labels are absent or the workload shifts, the threshold interval certifies nothing.","fun_headline_variants_meta":{"raw":{"variants":["Agreement ≠ safety: honest quorums can certify invalid states","Honest-but-wrong quorums: agreement is not validity","Epistemic faults: when honest quorums certify false transitions","Consensus isn't correctness when validators share wrong beliefs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000959,"raw_usage":{"total_tokens":3981,"prompt_tokens":860,"completion_tokens":3121,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":3060}},"tokens_in":604,"tokens_out":3121,"duration_ms":20227,"temperature":1.0,"reasoning_tokens":3060,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T21:18:04.031466+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a labeled workload, calibrate eδ and uε, compute q from Theorem 6.4, then evaluate held-out invalid and valid tasks. If invalid transitions receive certificates noticeably more often than the claimed δ confidence, or valid transitions fail liveness more often than ε, the sufficiency guarantee collapses.","supporting_citations":[],"review_version":1}