{"id":"61854d82-7d3e-4958-8dfb-2a1088d2b40c","arxiv_id":"2501.03999","paper_version":4,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"WAPTS reweights Thompson Sampling draws by empirical success rate to favor high-performing treatments, claiming faster convergence to near-optimal alternatives in sparse educational experiments.","lead":"This paper proposes WAPTS, a modification of Thompson Sampling that reweights each treatment's posterior draw by its observed success rate. The authors argue this helps find good-enough treatments faster when data are sparse, and they test it only in simulations of educational rating tasks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"WAPTS's zero-weight rule can permanently discard the truly optimal arm after early failures, invalidating the central claim of reliable identification.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: WAPTS's weight becomes zero when the empirical success rate is zero, allowing the truly optimal arm to be permanently discarded after early negative outcomes. This is not a consensus disagreement but an internal flaw in the algorithm: the implemented selection rule cannot recover from bad luck, so the claimed benefits of reliable identification and fast convergence cannot hold on all simulation seeds. The paper is transparent about this failure mode, which is creditworthy, but transparency does not rescue the central claim. Independent support is weak: there is no formal verification, and the paper's own tables (e.g., Table 1, Table 3, Appendix F) show WAPTS does not consistently outperform TS on all metrics—especially at larger effect sizes, TS sometimes has higher power or lenient correctness. The theoretical regret bound in Appendix A assumes a burn-in phase that is absent from Algorithm 1, so it does not apply. A concrete simulation with K=2, a high θ*, and many replications would directly demonstrate the permanent-exclusion failure, which is the decisive test for the central claim. Given this, the REJECT verdict is appropriate and should remain unchanged.","tokens_in":14023,"tokens_out":5164,"duration_ms":46981,"concrete_test":"Run a simulation with K=2, θ_1=0.9 (optimal), θ_2=0.5, N=100, over 10,000 replications, using Algorithm 1 exactly as written (initialize ks=kf=[0,0], define r_k=0 when ks+kf=0). Record the fraction of replications where arm 1 receives zero allocations after its first failure, and compute the probability of correctly selecting arm 1 for WAPTS versus standard Thompson Sampling. If the fraction of permanent-exclusion replications is non-negligible (e.g., >5%) or WAPTS's correct-selection probability is not above TS's, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that WAPTS reliably identifies near-optimal treatments in sparse settings is undermined by a structural property of Algorithm 1 (Appendix C). The allocation score is ω_k = (1 + r_k) r_k \\tilde{p}_k, where r_k = ks[k]/(ks[k]+kf[k]). If an arm has any failures and zero successes, r_k = 0 and ω_k = 0, so the arm can never be selected again via arg max. Consequently, if the truly optimal arm receives a failure on its first or early pulls—which occurs with positive probability for any Bernoulli reward with θ*<1—it is permanently discarded. The paper explicitly acknowledges this failure mode in the Discussion ('WAPTS currently updates in a direction that is difficult to reverse... could risk prematurely discarding promising treatments') and in Appendix A.4, yet Algorithm 1 contains no safeguard; the proposed reset mechanism is not implemented. This directly contradicts the claimed 'higher probability of correctly selecting near-optimal treatment' and 'faster posterior convergence to near-optimal treatments' because on seeds where the optimal arm is unlucky early, WAPTS never samples it again, making correct selection impossible. Furthermore, the regret analysis in Appendix A assumes a burn-in phase (uniform sampling for the first b rounds) that is absent from Algorithm 1, so the O(log n) bound does not apply to the implemented policy. Together, these issues invalidate both the empirical and theoretical support for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Weighted Allocation Probability Adjusted Thompson Sampling (WAPTS), a variant of Thompson Sampling in which each arm's posterior draw is multiplied by an empirical-success weight (1+r_i)r_i, with the goal of improving identification of near-optimal treatments under data-sparse educational experiments. The authors evaluate WAPTS against Thompson Sampling and uniform random allocation through simulations in learnersourcing and related settings, claiming faster convergence, higher probability of selecting near-optimal treatments, and comparable hypothesis-testing stability. The manuscript also provides a lenient-regret framework and a theoretical regret sketch in the appendix.","tokens_in":14450,"tokens_out":3120,"duration_ms":30411,"significance":"If the central claims were fully supported, WAPTS would be a simple, practical heuristic for adaptive experiments with many treatments and small samples, which is a genuinely relevant problem in educational technology. The paper's strength is its clear focus on lenient regret and on identifying sets of good-enough treatments rather than a single winner, and it includes extensive simulation tables across several application-inspired configurations. However, the evidence presented is internally mixed: several reported results contradict the abstract's 'consistently outperforms' claim, and the algorithm's zero-weight rule can permanently discard the truly optimal arm after early bad luck. These issues directly undermine the paper's main conclusions as currently stated.","major_comments":[{"comment":"The WAPTS selection score is defined as ω_i = (1 + r_i) r_i \\tilde{p}_i, where r_i = ks[i]/(ks[i]+kf[i]). If an arm has at least one failure and zero successes, then r_i = 0 and the weight is exactly 0, so the arm can never be selected again via arg max. Because any Bernoulli arm with θ_i < 1 has positive probability of an initial failure, the truly optimal arm can be permanently discarded with positive probability. The paper itself acknowledges this in the Discussion ('W APTS currently updates in a direction that is difficult to reverse... could risk prematurely discarding promising treatments') and in Appendix A.4, yet Algorithm 1 contains no safeguard such as forced exploration, a reset mechanism, or an additive epsilon. This structural property invalidates the claims of 'higher probability of correctly selecting near-optimal treatment' and 'faster posterior convergence to near-optimal treatments' on exactly the unlucky seeds where a good treatment fails early, and it also means the empirical averages over replications are not representative of reliable identification.","section":null},{"comment":"The abstract states that 'WAPTS consistently outperforms traditional TS' on key metrics, but the paper's own results contradict this. In Table 1, at effect sizes 0.20 and 0.30, TS has higher power than WAPTS for detecting at least one ϵ-lenient treatment (39.6% vs 38.1% and 78.3% vs 74.1%). In Appendix F, TS matches or exceeds WAPTS on mean outcome in several configurations: Table 6 (TS 0.617 vs WAPTS 0.611), Table 8 (both 0.565), and Table 9 (both 0.512). The text in the Power Trends section even states that 'at larger effects, TS more often identifies the true best treatment,' which is opposite to Table 1's true-best column (WAPTS 24.8% vs TS 16.5% at ∆=0.20, and 57.5% vs 43.6% at ∆=0.30). This internal inconsistency means the paper's headline claim of consistent superiority is not supported by the reported evidence; the authors need to either revise the claim to a more conditional statement or re-analyze the data.","section":null},{"comment":"The regret analysis assumes a burn-in phase of b rounds with uniform sampling ('For the first b rounds, W APTS selects treatments uniformly'), but Algorithm 1 has no such burn-in. The O(log n) regret bound is therefore derived for a different policy than the one implemented and evaluated. Additionally, the proof sketch relies on concentration inequalities applied to the reweighted scores w_k = (1 + r_k) r_k \\tilde{p}_k without providing a justification that these inequalities hold uniformly for the data-dependent weighting; the sketch states that 'using standard concentration inequalities' the mistake probability is bounded, but the reweighting factor itself depends on the same empirical counts and can be zero for the optimal arm. Thus the theoretical support for the implemented algorithm is not established.","section":null}],"minor_comments":[{"comment":"There are repeated typos and inconsistent formatting: 'policys' should be 'policies', 'treatmentes' in Appendix A.1, and the policy name is written as both 'W APTS' and 'WAPTS'; these should be unified.","section":null},{"comment":"The sentence 'at larger effects, TS more often identifies the true best treatment' is inconsistent with Table 1, where WAPTS has higher power for the true best treatment at ∆=0.20 and ∆=0.30; please correct the text or the table.","section":null},{"comment":"The caption refers to a 'vertical dashed line marks burn-in,' but Algorithm 1 has no burn-in phase. Please clarify whether the simulations included a burn-in period and, if so, why it is not in the pseudocode.","section":null},{"comment":"The correct-assignment rates are extremely low (around 2–5%) for all policies in the K=50 learnersourcing setting, and the mean-outcome differences between policies are very small; the text should discuss whether the reported 'superior counts' are practically meaningful given these near-chance rates.","section":null}],"recommendation":"reject","confidential_remarks":"The manuscript has a serious algorithmic flaw that the authors explicitly acknowledge but do not fix, and the empirical evidence does not support the abstract's central claim of consistent superiority. The paper would need a substantially different algorithm (e.g., with forced exploration or a reset mechanism), a revised theoretical analysis that matches the implemented policy, and a more careful, conditional framing of the empirical results. These are not local presentation fixes, so I recommend rejection rather than minor or major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper introduces WAPTS, a Thompson Sampling variant that multiplies each posterior draw by (1+r)*r, where r is the empirical success rate. That weighting function is new, and the application to educational learnersourcing with lenient regret is a genuine, well-motivated setting. The paper is also unusually candid: it acknowledges in the Discussion and Appendix A.4 that the algorithm can downweight a truly optimal treatment to near-zero allocation after early negative outcomes, and that unbounded regret is possible without a safeguard. That honesty counts for something.\n\nWhat the paper does well: the simulations are extensive, covering multiple sample sizes, effect sizes, gap structures, and application-inspired scenarios. The posterior dynamics plots and the majority-commit analysis give a clear picture of how the policies behave. The writing is straightforward, and the relationship to prior work (Merlis and Mannor, Rafferty et al.) is stated plainly.\n\nThe soft spots are load-bearing. The abstract claims WAPTS 'consistently outperforms' TS, but Table 1 shows TS has higher power for detecting at least one ϵ-lenient treatment at effect sizes 0.20 and 0.30, and Appendix F shows TS matching or beating WAPTS on mean outcome in several configurations. The zero-weight problem is structural, not a rare edge case: with Bernoulli rewards, any arm can draw its first pull as a failure, and if that happens for the true best arm, r=0 and the weight ω=(1+0)*0*p=0, so the arm is never selected again. Correct selection of the best treatment is then impossible on those seeds. The regret analysis in Appendix A assumes a uniform-sampling burn-in phase that is not present in Algorithm 1, so the O(log n) bound does not apply to the implemented policy. These are not minor gaps; they undermine both the empirical and theoretical support for the central claim.\n\nThe paper deserves a serious referee, because the idea is clear, the problem is real, and the flaws are fixable in principle (e.g., nonzero minimum weight, forced exploration). But it should not be accepted as is. The authors need to either revise the algorithm so the failure mode cannot occur, or drastically soften the claims to match the mixed results. I would not cite the current version, except possibly as a cautionary example of how aggressive reweighting can break a bandit algorithm.","headline":"A simple reweighting of Thompson Sampling that looks practical in sparse-data settings, but its own experiments contradict the 'consistently outperforms' claim, and the zero-weight rule can permanently kill the true best arm.","tokens_in":655,"tokens_out":1057,"would_cite":false,"duration_ms":27325,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"WAPTS, a reweighted Thompson Sampling rule, identifies near-optimal treatments faster and more reliably than standard Thompson Sampling in data-sparse educational experiments, according to simulation studies.","keywords":["Thompson sampling","multi-armed bandits","adaptive experiments","data-sparse settings","educational platforms","learnersourcing","lenient regret","posterior weighting"],"falsifier":"Simulate a fixed-gap bandit with $K=10$ arms, $N=239$ participants, and effect size $\\Delta=0.1$ across many seeds, with the optimal arm forced to receive three failures before any success in early rounds; measure the fraction of runs in which WAPTS reduces the optimal arm's allocation to zero for the rest of the horizon. If that fraction is not negligible, the claim that WAPTS reliably identifies near-optimal treatments fails for unlucky early outcomes.","tokens_in":13815,"feed_emoji":"🎓","tokens_out":5125,"duration_ms":42876,"temperature":0.7,"pith_summary":"The paper argues that standard Thompson Sampling wastes scarce student evaluations in educational experiments with many treatment arms and few participants, and proposes WAPTS (Weighted Allocation Probability Adjusted Thompson Sampling) to fix that. WAPTS adds a deterministic weight based on each arm's empirical success rate to the usual Thompson draw, pulling allocation toward treatments that are already looking good while still sampling from posteriors. Across simulation studies, the authors find WAPTS selects a near-optimal treatment with higher probability, converges its posterior faster, and keeps hypothesis-testing power comparable to Thompson Sampling. The motivation is practical: if true, instructors running small adaptive experiments could identify a small set of 'good enough' learning materials without needing large samples.","feed_headline":"Reweighted bandit rule finds near-optimal treatments on sparse data","feed_subtitle":"Simulations show the WAPTS rule selects near-optimal treatments sooner than standard Thompson Sampling with similar testing power.","key_machinery":"The central object is the weighted allocation score $\\omega_i = (1 + r_i) r_i \\tilde p_i$, defined for each treatment $i$. Here $\\tilde p_i$ is drawn from the Beta posterior $\\mathrm{Beta}(k_s[i]+1, k_f[i]+1)$, and $r_i = k_s[i]/(k_s[i]+k_f[i])$ is the empirical success rate. Multiplying the Thompson sample by an increasing function of the observed success rate reorders the draws: an arm with 80% observed success gets a larger score than one at 20% even when their posterior samples are equal, so allocation shifts toward empirically strong treatments. This is the mechanism that carries the paper's claim of faster convergence; it is also the mechanism that can in rare cases drive a truly optimal treatment's score to zero if early outcomes are negative.","core_discovery":"In the paper's own terms, WAPTS is a bandit policy that outperforms traditional Thompson Sampling in data-sparse educational settings on three metrics: higher probability of correctly selecting a near-optimal treatment, faster posterior convergence to near-optimal treatments, and comparable stability in hypothesis testing. The mechanism is a weighted allocation score $\\omega_i = (1 + r_i) r_i \\tilde p_i$, where $r_i$ is the empirical success rate and $\\tilde p_i$ is a Thompson sample from the Beta posterior; after scoring all arms, the policy plays the arm with the largest $\\omega_i$. Because the weight grows with empirical success, poor treatments are discarded earlier and promising treatments are exploited sooner, trading asymptotic optimality for faster practical discovery. The authors emphasize this is intended for 'lenient regret' settings where several treatments within an $\\epsilon$-band of the best are acceptable, which they argue matches how educators actually choose content.","pith_inferences":["Editorial extension: the same outcome-sensitive reweighting idea could be applied to contextual bandits or continuous outcomes, as the authors note, but the binary-success weight would need replacing with an effect-size-aware penalty.","Editorial extension: a reset mechanism of the kind proposed could be implemented cost-free and tested in the same simulations; if reset thresholds are chosen well, it may recover the rare-case failures without losing the early-exploitation gains.","Editorial extension: the paper's simulations compare WAPTS with Thompson Sampling and uniform random assignment only; a direct comparison with UCB-style or other probability-matching rules under the same data-sparse settings would sharpen the claim that the weighting, not just adaptivity, drives the improvement.","Editorial extension: because the weight is deterministic and monotone, WAPTS effectively turns Thompson Sampling into a more exploitative rule; one testable prediction is that its advantage grows as the effect size $\\Delta$ grows at small-to-moderate values, but reverses for very large $\\Delta$ where Thompson Sampling's exploration pays off."],"forward_implications":["In learnersourcing tasks where students rate peer-generated examples, WAPTS can surface a small set of useful materials after fewer ratings than Thompson Sampling, under the simulation conditions reported.","Practitioners can set the lenient tolerance $\\epsilon$ from sample size, number of treatments, and effect-size estimates rather than choosing it arbitrarily.","For data-sparse deployments (e.g., $N=50$ with $K=2$, $N=300$ with $K=50$), WAPTS shows higher correct-selection rates than uniform random assignment and usually than Thompson Sampling, though absolute correct-selection rates remain low when $K/N$ is large.","The theoretical $O(\\log n)$-type regret bound requires an additional safeguard like forced exploration or allocation resets; without it, WAPTS can in principle suffer unbounded regret in a rare unlucky run."],"supporting_citations":[{"why":"Supplies the Thompson Sampling regret analysis that WAPTS's proof sketch adapts for its theoretical bound.","marker":"Agrawal and Goyal 2012"},{"why":"Provides empirical evidence that Thompson Sampling is robust to noisy and delayed feedback, the baseline behavior WAPTS modifies.","marker":"Chapelle and Li 2011"},{"why":"Defines lenient regret, the evaluation criterion WAPTS is specifically designed to optimize.","marker":"Merlis and Mannor 2021"},{"why":"Documents challenges and underperformance of adaptive designs in small-sample educational experiments, the problem WAPTS addresses.","marker":"Williams et al. 2021"},{"why":"Shows posterior bias and allocation dynamics in adaptive educational experiments, supporting the motivation for reweighting.","marker":"Rafferty, Ying, and Williams 2019"},{"why":"Frames learnersourcing, the case-study domain where WAPTS is evaluated.","marker":"Khosravi et al. 2023"}],"fun_headline_variants":["WAPTS reweights bandit picks to find good treatments faster","Sparse-data bandit rule beats Thompson Sampling for learning content","WAPTS accelerates identification of near-optimal educational treatments","Weighted Thompson variant wins on sparse data for adaptive learning","New bandit policy finds promising content sooner with sparse data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The algorithm assumes that a treatment's observed success rate over the first few students is informative enough to reweight by, so that downweighting an unlucky arm toward zero cannot permanently hide the truly best treatment.","fun_headline_variants_meta":{"raw":{"variants":["WAPTS reweights bandit picks to find good treatments faster","Sparse-data bandit rule beats Thompson Sampling for learning content","WAPTS accelerates identification of near-optimal educational treatments","Weighted Thompson variant wins on sparse data for adaptive learning","New bandit policy finds promising content sooner with sparse data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000855,"raw_usage":{"total_tokens":3683,"prompt_tokens":881,"completion_tokens":2802,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":2728}},"tokens_in":497,"tokens_out":2802,"duration_ms":17904,"temperature":1.0,"reasoning_tokens":2728,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:41:19.594960+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate a fixed-gap bandit with $K=10$ arms, $N=239$ participants, and effect size $\\Delta=0.1$ across many seeds, with the optimal arm forced to receive three failures before any success in early rounds; measure the fraction of runs in which WAPTS reduces the optimal arm's allocation to zero for the rest of the horizon. If that fraction is not negligible, the claim that WAPTS reliably identifies near-optimal treatments fails for unlucky early outcomes.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides empirical evidence that Thompson Sampling is robust to noisy and delayed feedback, the baseline behavior WAPTS modifies."},{"cited_title":"Lenient Regret for Multi-Armed Bandits","cited_arxiv_id":"2008.03959","evidence_quote":"Defines lenient regret, the evaluation criterion WAPTS is specifically designed to optimize."},{"cited_title":"Challenges in Statistical Analysis of Data Collected by a Bandit Algorithm: An Empirical Exploration in Applications to Adaptively Randomized Experiments","cited_arxiv_id":"2103.12198","evidence_quote":"Documents challenges and underperformance of adaptive designs in small-sample educational experiments, the problem WAPTS addresses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows posterior bias and allocation dynamics in adaptive educational experiments, supporting the motivation for reweighting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Frames learnersourcing, the case-study domain where WAPTS is evaluated."}],"review_version":1}