{"id":"7c06d713-84b6-429a-b80e-54811076921a","arxiv_id":"2608.12831","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":1,"one_line_summary":"TCAB couples multiple adaptive policies on a tree, shares rewards when their complete context-action pairs match, and preserves each policy's standalone trajectory law at cost T plus cumulative edge disagreement.","lead":"This paper introduces Tree-Coupled A/B Testing (TCAB), a way to compare several adaptive decision policies at once while reusing rewards whenever the policies make the same decision on the same context. It proves each policy keeps exactly the trajectory distribution it would have if run alone, so the saved queries come without biasing the comparison.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified.","rationale":"The reader's weakest assumption matches mine: Assumption 3.2 is the operational premise on which the feedback-sharing construction rests. I agree with the reader that the main proofs are checkable and contain no load-bearing gap. The paper explicitly limits its optimality claims to conditionally exact edge-local designs and myopic tree optimality, and it flags the pilot-data and extrapolation limitations in the experiments. Lack of released code and an under-specified MST pilot weaken reproducibility but not the central theoretical claim. Therefore the ACCEPT verdict stands unchanged.","tokens_in":19475,"tokens_out":25552,"duration_ms":280918,"concrete_test":"Implement TCAB for T=5000 with J=3 history-dependent policies (e.g., epsilon-greedy, LinUCB, Thompson sampling) and a predictably time-varying tree; run 1000 replications. On each replication verify the pathwise identity N(T)=T+sum_{t,e}D_{e,t}. Then compare each policy's empirical trajectory distribution (context, action, reward sequences) against 1000 standalone runs using two-sample KS tests at the 5% level. If any policy shows a significant deviation, the exactness claim fails; otherwise the central theorem is confirmed in a nontrivial regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"I checked the main proofs and found no load-bearing gap. Theorem 5.1 follows by induction from the conditional marginal equality (23)-(24); the tree gluing lemma is standard and the one-sided rejection sampler realizes a maximal coupling. Theorem 5.2's pathwise identity and conditional edge-local lower bound are valid, and the lower bound does not require independence across edges. Theorem 5.3's regret-to-cost argument correctly handles small gaps through F_Delta(epsilon), and Corollary 5.4 follows by balancing the two terms. The only point on which the practical savings rest is Assumption 3.2: side-effect-free policy evaluation on unlabeled contexts and free rejected proposals. This is an explicitly stated scope condition, not a gap in the argument. If evaluating a policy on an unlabeled context is itself expensive or if rejection carries a real cost, the reward-query savings may not translate into total cost savings, but the formal claims are conditional on that premise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Tree-Coupled A/B Testing (TCAB), a round-synchronous experimental design that compares J history-dependent contextual-bandit policies while sharing reward queries across policies. At each round a predictable rooted tree connects the policies; each tree edge is assigned a maximal coupling of the two policies' complete context-action laws, and one reward is queried per connected component of matched edges. The central formal claims are: Theorem 5.1, that every policy's finite-horizon trajectory under TCAB has exactly its standalone distribution; Theorem 5.2, that the reward-query count satisfies N(T)=T+sum_{t,e} D_{e,t} pathwise and E[N(T)]=T+sum E[delta_{e,t}], with a conditional optimality result for conditionally exact edge-local designs on the selected tree; Corollaries 5.2-5.3 for star and minimum-spanning-tree designs; Theorem 5.3, that sublinear pseudo-regret of every policy plus almost-sure uniqueness of the oracle action imply E[N(T)]=T+o(T); and Theorem 5.4, finite-sample variance bounds for pairwise contrasts. Experiments on RewardBench, MMLU-Pro, and an MSLR-based adaptive search task compare TCAB.STAR and TCAB.MST with independent A/B/n baselines.","tokens_in":19586,"tokens_out":10391,"duration_ms":117793,"significance":"If the claims hold, the paper makes a substantial contribution to online experimentation and policy comparison. The key difficulty is real: for J>=3, pairwise maximal couplings are not generally jointly compatible, and for adaptive contextual policies, reuse of feedback can change later decisions. The tree-gluing construction resolves both obstructions in a transparent and implementable way. The cost identity N(T)=T+cumulative tree-edge TV is clean, falsifiable, and directly diagnostic of when feedback sharing helps. The paper is careful to scope its optimality claims as conditional, edge-local, and myopic, and it explicitly identifies Assumption 3.2 (side-effect-free policy evaluation with rejectable context proposals) as the operational premise behind the query-count savings. The main theorems are proved from first principles, with the one-sided rejection sampler connecting the abstract couplings to a practical algorithm. The regret-to-cost bridge in Theorem 5.3 is a genuinely useful principle that connects online-learning rates to experimental cost.","major_comments":[],"minor_comments":[{"comment":"The empirical cost-precision comparisons report only realized reward-query counts N(T); because Proposition 4.1 shows that the expected residual-proposal overhead can be as large as (J-1)T, the experiments would be more informative if they also reported realized proposal counts, or if all empirical claims were explicitly restricted to reward-query cost. As written, a reader could overinterpret the cost savings as total operating-cost savings in applications where policy evaluation itself is expensive.","section":"Section 6 and Figures 2-3"},{"comment":"The variance bound (27) relies on the cross terms E[C_s C_t] vanishing for s<t, which follows from the martingale-difference property of C_t with respect to the natural filtration. This property is not stated explicitly; I recommend adding one sentence defining the filtration and noting that E[C_t | F_{t-1}] = 0.","section":"Section 5.4, proof of Theorem 5.4"},{"comment":"The definition of U_{pv,t} counts only residual-branch proposals after the initial inheritance attempt, so the statement that the unconditional expected proposal contribution is at most one can be misread as a bound on total context draws. A clarifying sentence that this excludes the parent-context screening and the root draw would prevent confusion.","section":"Section 4.2, Proposition 4.1"},{"comment":"The sentence 'Almost-sure uniqueness is needed only to force F_Delta(epsilon) -> 0' is correct, but it would be clearer to note explicitly that this holds because finite K and positivity of the minimal gap imply P{Delta(X) > epsilon} -> 1 for every epsilon > 0.","section":"Section 5.3, Theorem 5.3"},{"comment":"The caption uses 'run in phases per round' while the text uses 'rounds' and 'depths'; aligning the caption terminology with Algorithm 1 would improve readability.","section":"Figure 1 caption"}],"recommendation":"minor_revision","confidential_remarks":"No confidential concerns regarding citation patterns or scope. The paper fits the journal's scope in online experimentation and sequential decision-making; the Assumption 3.2 limitation should be kept prominent in the final version but does not affect the validity of the formal results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper earns its central claims. TCAB couples J>=3 adaptive policies through a tree of maximal couplings, so every policy's standalone trajectory law is preserved exactly, and the reward-query count satisfies the pathwise identity N(T)=T+sum of edge-mismatch indicators. The extension from two context-free policies to J>=3 contextual ones is the real novelty—pairwise maximal couplings don't play well together, and the tree gluing lemma is the right fix. The cost identity and the edge-local optimality theorem are proven cleanly; the lower bound is a genuine coupling argument, not a tautology. The regret-to-cost result (T+o(T) queries under sublinear regret) is a nice theoretical payoff and the variance bounds for contrasts look sound. I checked the main proofs and found no load-bearing gaps.\n\nWhat's new here is the exactness-plus-cost-identity package: previous work (Artificial Replay) only handled two context-free policies, and the contextual, history-dependent case needed the complete context-action pair coupling, not just arm replay. The paper is honest about the scope: the operational Assumption 3.2 is that you can evaluate policies on unlabeled contexts for free, reject proposals without side effects, and count only reward queries. That assumption is explicit and reasonable in many LLM-eval and recommendation settings, but it does mean the practical savings may not survive if policy evaluation itself is costly. The authors say this clearly.\n\nSoft spots are smaller than the claims. The TCAB.MST variant depends on a pilot-estimated similarity score whose construction is under-specified; no code is released; and the experiments never compare against the closest prior method (two-policy Artificial Replay), so the J>=3 advantage is only shown against independent baselines. These are fixable in revision rather than fatal. The variance proof is a bit compressed in its martingale-difference step, though the bounds themselves check out.\n\nMy read: the paper is for researchers and practitioners who run multi-policy comparisons with expensive outcomes—LLM eval, search, online experiments. It's a clear, proof-backed method with a precise condition on when it helps. It deserves a serious referee; I'd accept after moderate revision (code, MST details, a comparison to the two-policy baseline). Not a field-reshaping paper, but a solid and useful one.","headline":"A genuinely new, provably exact method for multi-policy feedback sharing; the central theorems are solid and the paper deserves a real referee.","tokens_in":20125,"tokens_out":2091,"would_cite":true,"duration_ms":20953,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tree-Coupled A/B Testing lets any number of adaptive policies share rewards along the edges of a predictable tree, preserving each standalone trajectory law exactly while cutting query count from JT to T plus cumulative total-variation…","keywords":["online experimentation","A/B testing","contextual bandits","maximal coupling","feedback sharing","adaptive policy comparison","total variation distance","tree coupling"],"falsifier":"Take a one-round problem with two deterministic policies over a single context, where the two policies choose the same action with known probability $1-\\delta$. TCAB should query exactly one reward with probability $1-\\delta$ and two rewards with probability $\\delta$, so the expected cost is $1+\\delta$, and each policy's action marginal must equal its standalone Bernoulli law. A run that deviates from either assertion beyond Monte Carlo noise disproves the cost identity or the exact-trajectory claim.","tokens_in":19230,"feed_emoji":"🌳","tokens_out":8637,"duration_ms":80640,"temperature":0.7,"pith_summary":"Comparing J candidate policies in a live experiment usually costs JT reward-bearing interactions because each policy runs its own horizon-T trajectory. This paper introduces Tree-Coupled A/B Testing (TCAB), which couples the policies' complete context–action pairs along the edges of a predictable tree and lets matched policies share one realized reward. The central claim is that every policy still has exactly the trajectory law it would have had running alone, while the expected number of reward queries is T plus the cumulative total-variation distance between parent and child pairs on the tree. For policies that increasingly agree on the optimal action, the whole comparison costs $T+o(T)$ queries instead of $JT$. That matters whenever each interaction is costly, risky, or scarce.","feed_headline":"Tree-coupled A/B tests use T plus mismatch queries, not JT","feed_subtitle":"Every candidate policy keeps its standalone behavior exactly, while matched decisions share one reward instead of J.","key_machinery":"The central object is the tree coupling: at each round $t$, a predictable rooted tree connects the $J$ policies, and every parent–child pair $(p,v)$ is matched by a maximal coupling of the complete-pair laws $\\nu_{v,t}^{h_v}(dx,da)=P_X(dx)\\,\\pi_{v,t}(a\\mid h_v,x)$. The tree gluing lemma shows that prescribed couplings on the edges of an acyclic graph can always be glued into one joint law preserving each node's marginal, which is what lets $J-1$ pairwise maximal couplings coexist even though full pairwise maximality is impossible for $J\\ge 3$. The cost identity $N(T)=T+\\sum_{e,t}D_{e,t}$ follows because each matched component needs only one reward. The implementation primitive is a one-sided rejection sampler that accepts the parent's context–action pair with probability $\\min(1,\\pi_v(a_p\\mid h_v,x_p)/\\pi_p(a_p\\mid h_p,x_p))$, so the context density cancels.","core_discovery":"TCAB constructs, at each round, a rooted spanning tree over the J policies whose choice is measurable with respect to past histories, then glues pairwise maximal couplings of the complete context–action laws along the tree edges. Because the graph is a tree, the $J-1$ prescribed edge couplings can always be combined without distorting any node's marginal. Whenever a child's complete pair equals its parent's, the child inherits the parent's reward draw; otherwise it opens a new lineage. The paper proves (Theorem 5.1) that every policy's full horizon-T history has the standalone distribution and (Theorem 5.2) the pathwise identity $N(T)=T+\\sum_{t,e}D_{e,t}$, with expectation $T+\\sum_t \\mathbb{E}[\\delta_{e,t}]$, which is the minimum possible among conditionally exact edge-local designs on the selected tree. A one-sided rejection sampler implements the coupling using only action-probability ratios, so the context density and reward kernel never need to be known.","pith_inferences":["The paper proves myopic optimality of the MST rule but does not claim horizon-global optimality; a natural extension is to ask whether trees chosen by predicted future disagreement can beat any greedy sequence.","When the coupling-access assumption holds but reward queries are destructive or irreversible, TCAB could make comparisons safer by spending nearly one trajectory's worth of outcomes rather than J copies of it.","The same tree-coupling mechanism could be combined with logged replay: rejected context proposals carry information that a separate replay evaluator could store without disturbing the coupling's exactness, although the paper does not develop this."],"forward_implications":["For $J$ related policies, the reward-bearing cost drops from $JT$ to $T$ plus cumulative tree-edge total variation, making multi-policy comparison feasible when outcomes are expensive.","Each policy's finite-horizon trajectory law is exactly its standalone law, so all cumulative values and pairwise baseline contrasts are unbiased.","If every policy has sublinear pseudo-regret and the oracle action is unique almost surely, the expected query cost is $T+o(T)$.","Pairwise contrast variance is controlled: only mismatched edges along the tree path between two policies and the realized regret variances of the two policies appear in the bound.","A baseline-centered star and a current-round minimum-spanning tree give explicit designs, with the MST minimizing current-round expected cost among edge-local tree designs."],"supporting_citations":[{"why":"Supplies the canonical maximal-coupling construction and the total-variation lower bound on disagreement probability used for each tree edge.","marker":"[Thorisson, 2000]"},{"why":"Provides the coupling background for the two-marginal maximal coupling that underlies the edge sampler.","marker":"[Lindvall, 2002]"},{"why":"Shows that all pairwise maximal couplings need not be compatible for J≥3, motivating the tree restriction and gluing argument.","marker":"[Angel and Spinka, 2019]"},{"why":"Gives the one-sided rejection construction on which the exact implementation in Section 4.2 is based.","marker":"[Corenflos and Särkkä, 2022]"},{"why":"Introduces Artificial Replay for comparing two context-free bandit algorithms, the closest prior method that TCAB extends to contextual policies and J≥3.","marker":"[Meng et al., 2026]"},{"why":"Provides the exact replay evaluator for contextual bandits that motivates exact prospective coupling rather than off-policy weighting.","marker":"[Li et al., 2011]"},{"why":"Anchor for the stochastic bandit regret rates used in the regret-to-cost efficiency theorem.","marker":"[Auer et al., 2002]"},{"why":"Documents overlapping-experiment infrastructure and establishes the traffic-reuse versus feedback-reuse distinction the paper builds on.","marker":"[Tang et al., 2010]"}],"fun_headline_variants":["Tree-coupled A/B/n: exact comparisons with T + mismatch queries","A/B/n with shared rewards: every policy exact, cost T+o(T)","Tree-coupled A/B tests need T + edge mismatches, not JT","Exact A/B/n via tree coupling: T + mismatch queries in expectation","Reward sharing via tree coupling: A/B/n at T plus mismatch cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The experiment must be able to evaluate a policy's action distribution on an unlabeled context without spending a reward and without changing the policy's state, and to reject that context without side effects.","fun_headline_variants_meta":{"raw":{"variants":["Tree-coupled A/B/n: exact comparisons with T + mismatch queries","A/B/n with shared rewards: every policy exact, cost T+o(T)","Tree-coupled A/B tests need T + edge mismatches, not JT","Exact A/B/n via tree coupling: T + mismatch queries in expectation","Reward sharing via tree coupling: A/B/n at T plus mismatch cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001181,"raw_usage":{"total_tokens":4930,"prompt_tokens":1051,"completion_tokens":3879,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":667,"completion_tokens_details":{"reasoning_tokens":3780}},"tokens_in":667,"tokens_out":3879,"duration_ms":30015,"temperature":1.0,"reasoning_tokens":3780,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:26:45.137134+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a one-round problem with two deterministic policies over a single context, where the two policies choose the same action with known probability $1-\\delta$. TCAB should query exactly one reward with probability $1-\\delta$ and two rewards with probability $\\delta$, so the expected cost is $1+\\delta$, and each policy's action marginal must equal its standalone Bernoulli law. A run that deviates from either assertion beyond Monte Carlo noise disproves the cost identity or the exact-trajectory claim.","supporting_citations":[{"cited_title":"Lemma A.2(One-sided rejection sampler).Suppose samples fromν0,ν 1 and the density ratios are available","cited_arxiv_id":null,"evidence_quote":"Provides the coupling background for the two-marginal maximal coupling that underlies the edge sampler."}],"review_version":1}