{"id":"ce4627ad-c1f8-4187-beb2-ad1edec0a0e0","arxiv_id":"2607.04315","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"TaS-NDPO identifies the arm maximizing expected NDPO with δ-correctness and asymptotic optimality under interventional samples, via Track-and-Stop plus a cutting-set solver.","lead":"The paper gives a fixed-confidence bandit algorithm that finds the treatment maximizing the natural direct potential outcome, ignoring effects that travel through a mediator. This matters for settings like advertising or medicine where some pathways are undesirable and should not count toward success.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Theory assumes exact optimizers of the bi-convex inner problem; practice uses alternating minimization that only reaches stationary points.","rationale":"The Reader correctly flags the untestable Assumption 1 and the exact-oracle gap, but treats them as residual caveats of roughly equal weight. Assumption 1 is standard for any mediation analysis and is already stated as a modeling premise; if it fails the algorithm simply optimizes a different (still well-defined) functional. The exact-versus-stationary gap is more internal: it sits between the statements of Theorems 3–4 and the algorithm that is coded and evaluated. Because the paper itself acknowledges the gap and the optimality proof never closes it, the concern is load-bearing for the strongest claim. The recommended concrete test isolates precisely that discrepancy on instances where T★ is known. The overall verdict remains CONDITIONAL; the test would decide whether the optimality half of the claim needs a stronger qualifier or can be left as is.","tokens_in":27799,"tokens_out":608,"duration_ms":6738,"concrete_test":"On the synthetic instances of Appendix E (K=3, M=3, known closed-form T★), replace the alternating-minimization oracle by a global solver (e.g., multi-start or spatial branch-and-bound) for every master/inner call; recompute the empirical limsup \tauδ/kl(δ,1-δ) as δ\to0. If the ratio systematically exceeds T★(P) under the paper’s solver but stays ≤T★ under the global solver, the optimality claim does not transfer to the implemented algorithm.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim (Theorems 3–4) is that TaS-NDPO is δ-correct and almost-surely asymptotically optimal for the arm maximizing θ. Both results rest on the plug-in allocation ŵ(t) and the GLRT Zt being computed from exact solutions of the reduced bi-convex program (23)–(26) / (50)–(51). The paper states explicitly (Section 6.2 and Appendix C) that the cutting-set method and alternating minimization are only guaranteed to produce a stationary point, not a global optimum, and that “our theoretical analysis … assumes access to the exact optimizer.” If the stationary point is not global, the computed allocation need not converge to the unique characteristic w★(P) (Assumption 4 / Lemma 2) and the GLRT lower bound of Lemma 5 can fail, so the limsup claim of Theorem 4 need not hold for the algorithm that is actually run. This is the single most load-bearing gap between the proved statements and the implemented procedure.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper studies fixed-confidence best-arm identification of the treatment maximizing the expected natural direct potential outcome (NDPO) in a causal bandit model. Under Assumption 1 it identifies the expected NDPO by the interventional functional θ(x)=∑_z P_x(1|z)P_{x0}(z) (Theorem 1). It derives the corresponding instance-dependent lower bound on sample complexity (Theorem 2), then designs TaS-NDPO: a Track-and-Stop algorithm that uses cell-level forcing, competitor coverage and a cutting-set method to solve the resulting bi-convex semi-infinite allocation problem. The algorithm is proved δ-correct (Theorem 3) and almost-surely asymptotically optimal (Theorem 4) under additional regularity assumptions, and is evaluated on the IPinYou advertising data set and a framing data set.","tokens_in":28137,"tokens_out":1134,"duration_ms":28500,"significance":"If the claims hold, the work supplies a sample-efficient, high-probability method for selecting interventions according to direct effects that deliberately exclude mediated pathways—an objective of clear practical interest in advertising (positioning bias), medicine and policy. The technical core is a non-trivial adaptation of the Track-and-Stop framework to a non-separable alternative set induced by the nested counterfactual; the paper provides complete change-of-measure lower bounds, concentration arguments for the GLRT, and almost-sure asymptotic optimality, together with reproducible real-world experiments that demonstrate substantial sample-complexity gains over arm-level and uniform baselines. These contributions are of genuine interest to both the causal-bandits and fixed-confidence BAI communities.","major_comments":[{"comment":"The δ-correctness and almost-sure asymptotic optimality statements (Theorems 3–4, Lemmas 2 and 5) are proved under the explicit hypothesis that the reduced bi-convex program (23)–(26) / (50)–(51) is solved to global optimality. Section 6.2 and Appendix C state that the implemented cutting-set + alternating-minimization procedure is only guaranteed to reach a stationary point (Tseng 2001) and that “our theoretical analysis … assumes access to the exact optimizer.” Without a proof that every stationary point yields an allocation sufficiently close to w★(P) for the liminf growth of the GLRT to hold, or an approximate-optimality analysis of the cutting-set method, the optimality claim does not apply to the algorithm that is actually run and evaluated. This is a load-bearing gap between the proved statements and the implemented procedure.","section":"Section 6.2, Appendix C, Theorems 3–4"},{"comment":"Identification of the expected NDPO by θ(x) rests entirely on the untestable Assumption 1 (Y_{x,z} ⊥ Z_{x'} for all x,x',z). While the paper correctly notes the graphical d-separation characterisation, the central applied claim is about NDPO rather than merely θ. A short sensitivity discussion or numerical illustration of how violations of the independence affect the ranking of arms would strengthen the bridge from theory to the motivating applications.","section":"Section 4, Theorem 1, Assumption 1"}],"minor_comments":[{"comment":"The forcing schedules h(t)=⌈t^a⌉ and g(t) are free parameters. The specific numerical choices used for the IPinYou and framing experiments (and any sensitivity checks) should be reported in the main text or Appendix E.","section":"Algorithm 1, Section 8"},{"comment":"Empirical error rates in Tables 1–2 are estimated from only 100 runs; while consistent with δ=0.05 they do not strongly corroborate the finite-sample guarantee. A larger Monte-Carlo study or a plot of empirical error versus δ would be useful.","section":"Tables 1–2"},{"comment":"Laplace smoothing with α=0.05 is used throughout the experiments but is not mentioned in the theoretical analysis; a brief remark on its effect on the plug-in estimators would improve clarity.","section":"Section 8"},{"comment":"Minor notational inconsistencies appear (e.g., δ-correctness is used in the abstract before definition; occasional switches between P and R for models). A careful pass would eliminate them.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is carefully written and the core technical development is sound once the exact-optimizer hypothesis is granted. The single most important revision is to close (or carefully qualify) the theory–practice gap on the bi-convex solver; if that is done cleanly the paper would be a strong accept for a top ML/stats venue. Scope and novelty are appropriate."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This paper gives a clean fixed-confidence BAI method for the arm that maximizes expected natural direct potential outcome under pure do(X) sampling. That is the useful bit: they identify θ(x) = ∑_z Px(1|z)Px0(z), derive the instance-dependent lower bound that accounts for the baseline-mediator coupling, and build a Track-and-Stop variant (cell-level forcing + competitor coverage + cutting-set) whose asymptotic optimality matches the bound under the stated assumptions.\n\nWhat is new is not TaS or NDPO themselves, but the non-separable alternative set, the three-arm reduction, and the cell-level information control that classical arm-level TaS does not need. The proofs (identification, lower bound, δ-correctness, almost-sure asymptotic optimality) are standard change-of-measure and concentration arguments and look carefully done. The IPinYou experiments are on a real public advertising log and show roughly 50% median sample reduction versus the arm-level variant and uniform baselines, with zero empirical error at δ=0.05. That is concrete evidence the design choices matter.\n\nSoft spots, in proportion. Assumption 1 (no unmeasured Z–Y confounding) is untestable from interventional data alone; if it fails, the algorithm optimizes the wrong functional. That is the usual causal-identification caveat and they state it clearly. More load-bearing for the theory-practice link: the optimality proofs assume exact solutions of the bi-convex inner problem, while the implementation uses alternating minimization that only guarantees a stationary point. The paper itself flags this. Empirically the method still works and the lower-bound scaling checks out on synthetic gaps, so the gap is real but not fatal. No released code is a minor annoyance for a methods paper. Free parameters (forcing schedules, Laplace α) are standard and not hidden.\n\nThis is for people working on causal bandits or sequential mediation. It deserves a serious referee. I would cite the lower-bound construction and the cell-level forcing idea if I were doing path-specific sequential design. Send it to peer review.","headline":"Solid TaS extension for NDPO maximization with matching lower bound and real-data gains; the exact-vs-stationary optimizer gap is real but does not sink the contribution.","tokens_in":28706,"tokens_out":547,"would_cite":true,"duration_ms":6329,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A Track-and-Stop algorithm finds the arm that maximizes the expected natural direct potential outcome with δ-correctness and asymptotic optimality.","keywords":["best-arm identification","causal mediation","natural direct potential outcome","Track-and-Stop","fixed confidence","causal bandits","sample complexity"],"falsifier":"On any instance where the true NDPO-optimal arm differs from the interventional-mean optimum, run TaS-NDPO with decreasing δ and check whether the empirical ratio of stopping time to kl(δ,1-δ) approaches the computed T*(P) while the recommended arm remains correct with frequency at least 1-δ; systematic under-performance or wrong recommendations would refute the optimality or correctness claims.","tokens_in":28719,"feed_emoji":"⚡","tokens_out":650,"duration_ms":6735,"temperature":0.7,"pith_summary":"The paper asks which treatment maximizes expected outcome while ignoring a pathway that runs through a mediator the researcher wants to exclude (for example, slot bias in ads or an adverse intermediate response in medicine). It first shows that this nested counterfactual quantity, the expected natural direct potential outcome, is identified from the interventional distributions that a causal bandit can actually sample. It then gives a fixed-confidence best-arm algorithm, TaS-NDPO, that allocates samples by solving a characteristic max-min problem with a cutting-set procedure and stops via a generalized likelihood-ratio test. The algorithm is proved δ-correct and almost-surely asymptotically optimal for the resulting instance-dependent lower bound. On a large advertising data set the method recovers a different creative than ordinary click-rate maximization and uses roughly half as many samples as an arm-level baseline.","feed_headline":"Bandit finds best treatment ignoring the mediator pathway","feed_subtitle":"TaS-NDPO is δ-correct and asymptotically optimal for the expected natural direct potential outcome","key_machinery":"The characteristic time T*(P) of the non-separable alternative set Alt(P), together with the three-arm reduction and cutting-set solution of the resulting bi-convex projection that yields the Track-and-Stop allocation and the cell-level forcing rule.","core_discovery":"Under the stated identification and regularity conditions, TaS-NDPO identifies the unique arm that maximizes θ(x)=∑_z Px(1|z)Px0(z) with probability at least 1-δ and satisfies limsup \tauδ/kl(δ,1-δ)≤ T*(P) almost surely, matching the information-theoretic lower bound derived for the same functional.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["TaS-NDPO finds max natural direct outcome arm in fixed confidence","Causal bandit identifies best treatment excluding mediator pathway","Track-and-Stop solves semi-infinite opt for NDPO best-arm ID","δ-correct TaS algorithm hits info-theoretic bound for expected NDPO","Sample-efficient best-arm ID for natural direct potential outcomes"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The claim that the expected natural direct potential outcome equals the identifiable functional θ rests on an untestable independence between the nested potential outcome and the mediator under any reference treatment; if unmeasured confounding between mediator and outcome exists, the algorithm optimizes the wrong quantity.","fun_headline_variants_meta":{"raw":{"variants":["TaS-NDPO finds max natural direct outcome arm in fixed confidence","Causal bandit identifies best treatment excluding mediator pathway","Track-and-Stop solves semi-infinite opt for NDPO best-arm ID","δ-correct TaS algorithm hits info-theoretic bound for expected NDPO","Sample-efficient best-arm ID for natural direct potential outcomes"]},"model":"grok-4.5","effort":"low","cost_usd":0.003948,"raw_usage":{"total_tokens":1186,"prompt_tokens":696,"num_sources_used":0,"completion_tokens":95,"cost_in_usd_ticks":39480000,"prompt_tokens_details":{"text_tokens":696,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":395,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":696,"tokens_out":95,"duration_ms":4221,"temperature":1.0,"reasoning_tokens":395,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T20:08:17.893309+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On any instance where the true NDPO-optimal arm differs from the interventional-mean optimum, run TaS-NDPO with decreasing δ and check whether the empirical ratio of stopping time to kl(δ,1-δ) approaches the computed T*(P) while the recommended arm remains correct with frequency at least 1-δ; systematic under-performance or wrong recommendations would refute the optimality or correctness claims.","supporting_citations":[],"review_version":1}