{"id":"fca14bc1-b462-47f0-ad63-490e5ac302af","arxiv_id":"2602.06014","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Optimistic modifications of Thompson sampling make each arm's pull count concentrate around a deterministic scale, yielding asymptotically valid Wald inference in K-armed Gaussian bandits with multiple optimal arms.","lead":"Two Thompson-sampling variants—one that inflates sampling variance and one that adds an optimism bonus—are proved to make arm-pull counts concentrate in K-armed Gaussian bandits, restoring valid Wald confidence intervals under adaptive data collection. This answers an open extension from two-armed to many-armed Thompson sampling and quantifies a modest regret price for inference.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Embedded-chain winner approximation in the proof of Theorem 4.1(a) is asserted without proof; q_i is a size-biased version of the single-round map.","rationale":"The reader correctly flags Lemma C.3 as an external dependency; that is a real concern about the suboptimal-count part of Theorem 4.1. However, the more load-bearing gap is the embedded-chain approximation in Appendix A.1.2. The paper's claimed novelty for K-armed bandits with multiple optimal arms rests on the Lyapunov analysis for the embedded optimal-pull process. The assertion that q_{i,τ_s} is close to g_i(e y_s) is not a consequence of Lemma A.3 and is not otherwise proved. The size-bias mechanism I describe is concrete, localizable, and would affect the central drift inequality. This does not demonstrate the theorem is false, but it leaves the central proof incomplete. The verdict remains CONDITIONAL, consistent with the reader's assessment, though the condition should explicitly include a proof or replacement of the embedded-chain approximation.","tokens_in":28287,"tokens_out":26177,"duration_ms":269031,"concrete_test":"Derive the exact expression for q_i on the embedded chain: q_i = g_i · E[P(max_sub < M_i | argmax=i)] / (Σ_j g_j E[P(max_sub < M_j | argmax=j)]), where M_i is the max optimal sample given argmax=i. Check whether the variance-dependent factors are all 1 + O(η_T) uniformly on Ω_T ∩ E_T. If not, compute the additional term in the Lyapunov drift and test whether it still vanishes; if it does not, the proof of Theorem 4.1(a) is incomplete. A quick numerical check with K=3, two optimal arms with imbalanced counts, and one suboptimal arm with small count would reveal whether the bias is non-negligible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To prove uniform allocation among optimal arms (Theorem 4.1(a)), Appendix A.1.2 embeds the process at the times τ_s of optimal pulls and defines q_{i,τ_s} = P(next optimal pull is arm i | F_{τ_s}). It then asserts, without proof, that on E_T ∩ Ω_T, max_i |q_{i,τ_s} - g_i(e y_s)| ≤ c m η_T, where g is the single-round pure-noise winner map used in the equal-means analysis. Lemma A.3, cited as 'Similarly', only bounds the effect of additive mean-shift perturbations on a single round's winner probabilities. The embedded q is not a single-round probability: the next optimal pull can occur after one or more suboptimal pulls, so q_i conditions on the event {an optimal arm is eventually pulled}. Since suboptimal samples are independent of optimal samples, q_i is the size-biased version q_i ∝ g_i · s_i with s_i = P(max_opt > max_sub | argmax_opt = i). The factors s_i are variance-dependent and are not shown to be within 1 ± O(η_T) of each other on Ω_T. If they differ, the Lyapunov drift inequality (A.9) for the embedded process fails, and no other argument establishes e V_{s1} →P 0. Thus the key step for multiple optimal arms is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies adaptive inference for K-armed Gaussian bandits under two optimistic variants of Thompson sampling: variance-inflated TS (Algorithm 1, Option B) and mean-bonus TS (Algorithm 1, Option C). It claims both are stable in the sense of Lai and Wei: optimal arms receive asymptotically T/m pulls, suboptimal arm a receives (2c(A) log T)/Δ_a^2 pulls where c(A)=σ(A) or β(A), and consequently studentized sample means are asymptotically normal and Wald confidence intervals are valid. The proofs introduce a pure-noise winner map and Lyapunov drift for uniform allocation among optimal arms, and geometric waiting-time arguments for suboptimal-pull asymptotics. The variance-inflation result is presented as resolving the K-armed extension question raised by Halder et al. (2025).","tokens_in":28663,"tokens_out":13600,"duration_ms":151195,"significance":"If the results are correct, the paper makes a useful contribution: it identifies optimism as a mechanism for restoring stability of Thompson sampling, treats the multiple-optimal-arm regime, and gives a clean route from stability to Wald inference. The equal-means winner-map analysis is an appealing technique, and the mean-bonus proof is largely self-contained. The claimed regret penalty is mild when the tuning parameters grow slowly. At the same time, the ``sharp constant 2'' is algorithm-relative: it is inherited from the Gaussian index distribution and depends on the inflation factor σ(A) or β(A), so it is not a parameter-free law. The central gap described below concerns the proof of uniform allocation among multiple optimal arms for variance-inflated TS, and must be repaired before the main theorem is established.","major_comments":[{"comment":"The key step for Theorem 4.1(a) asserts, after ``Similarly to Lemma A.3,'' that on E_T∩Ω_T, max_i |q_{i,τ_s} − g_i(e y_s)| ≤ c m η_T. This is not a consequence of Lemma A.3. Lemma A.3 bounds a single-round winner probability under additive mean shifts; q_{i,τ_s} is a conditional first-hitting probability: it requires that no optimal arm be pulled for possibly several steps and that the eventual optimal pull be arm i. Conditioning on the first optimal pull size-biases the within-optimal maximizer, since suboptimal index samples act as a random threshold. In particular q_i is proportional to g_i·s_i with s_i = P(M_opt > M_sub | argmax_opt = i), and the factors s_i are not shown to be within 1 ± O(η_T). Without this bound the Lyapunov drift recursion for Ṽ_s has no justification, so the claim of asymptotically uniform allocation among multiple optimal arms is unsupported.","section":"Appendix A.1.2, Eq. (A.9)"},{"comment":"The proof of Theorem 4.1(b) relies at a load-bearing point on Lemma C.3, imported from Halder et al. (2025), for the expected suboptimal-pull bound E[N_{a,T}] = O(σ(A) log(TΔ_a^2/σ(A))/Δ_a^2). This lemma is used to define the high-probability event Ω_T in (A.3) and to control the burn-in time in Lemma A.7. The manuscript neither proves this lemma nor states the scope of the original result (two-armed vs. K-armed). If the cited lemma is only two-armed, the K-armed variance-inflation result is incomplete. Please supply a proof for K arms or an explicit statement that the source lemma holds in the required K-armed setting.","section":"Appendix A, Lemma C.3 and Eq. (A.3)"}],"minor_comments":[{"comment":"The abstract promises ``independent sub-Gaussian reward noises,'' but Section 2.1 explicitly sets rewards to be Gaussian with unit variance, and the proofs use Gaussian tail bounds and Gaussian posterior indices throughout. The abstract should be aligned with the actual model, or the sub-Gaussian claim proved.","section":"Abstract and §2.1"},{"comment":"There are notation/typo inconsistencies: in the text immediately before Lemma C.4, α−(ε) is called (Δ_a − 3ε)^2/2 whereas Eq. (A.17) defines α−(ε)=(Δ_a + 3ε)^2/2; the displayed formula for n−(T) uses α+(ε) in one place and α−(ε) in another. In the proof of Lemma A.6, the displayed couplings 1{U≤p+} ≤ 1{A=a} ≤ 1{U≤p−} appear reversed if p+ denotes the upper bound and p− the lower bound. These should be corrected.","section":"§A.2.2, Lemma A.6 and Eq. (A.17)-(A.18)"},{"comment":"The overview says the final result is a bracket N_{a,T} = Θ(2σ(A) log T/Δ_a^2), but Theorem 4.1(b) is the stronger claim N_{a,T}/(σ(A) log T) → 2/Δ_a^2. The overview should state the convergence, not just order notation.","section":"§5.1, Step 3"}],"recommendation":"major_revision","confidential_remarks":"The mean-bonus part (Theorem 4.2) is much more self-contained and appears substantively sound to me. The main risk is Theorem 4.1(a) for the multiple-optimal-arm case: the embedded-chain approximation is not proven, and the imported Lemma C.3 is a second significant gap. These are fixable in principle, but they are central to the paper's advertised contribution, so I cannot recommend acceptance without a revised proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a serious attempt at a known open problem—extending stable TS to K arms—and the mean-bonus variant is a genuinely new mechanism. But I wouldn't bank on Theorem 4.1 as written. The equal-means analysis is clean and rigorous: the winner-map monotonicity, the dot-product inequality, and the Lyapunov drift are all properly handled. If you only read Section A.1.1, you get a nice self-contained proof of uniform allocation in the equal-means case. The mean-bonus proof (Theorem 4.2) also looks solid to me; the bracketing argument with the good event is explicit and doesn't lean on outside results.\n\nThe soft spot is the bridge from equal-means to general K-armed in Theorem 4.1(a). The embedded-chain argument asserts, on E_T ∩ Ω_T, that max_i |q_{i,τ_s} - g_i(ey_s)| ≤ c m η_T, and refers to Lemma A.3 as 'similarly.' But Lemma A.3 is a perturbation bound for additive mean shifts in a single round; it doesn't cover the presence of suboptimal arms with different means. The event Ω_T only bounds total suboptimal pulls; it does not make the per-step probability of a suboptimal pull negligible in the way the bound requires. So that step is unproven as it stands. The stress-test note frames this as a size-bias issue; I think the actual problem is even more basic—the paper's q_i is the probability that the immediate next action is arm i, not the probability that the next optimal pull is arm i, and the claimed closeness to the optimal-only winner map needs an argument that isn't there.\n\nSecond issue: the paper imports Lemma C.3 from Halder et al. for the suboptimal-pull expectation in K arms. If that lemma is two-armed only, then the good event Ω_T and the burn-in time bound in Lemma A.7 both rest on a scope that isn't stated. The authors need to either prove the K-armed version or clearly state where it comes from.\n\nMinor: the abstract in the arXiv metadata says sub-Gaussian noise, while the paper itself analyzes Gaussian rewards. Align that.\n\nOverall: the central claim is plausible and the proof has real structure, but Theorem 4.1 is not complete as written. I'd send it to a serious referee, asking them to focus on the embedded-chain approximation and the scope of Lemma C.3. The mean-bonus half might survive a revision; the variance-inflation half needs more work.","headline":"Plausible K-armed stability result, but the key transfer step for multiple optimal arms is asserted rather than proven; worth refereeing.","tokens_in":29104,"tokens_out":7905,"would_cite":false,"duration_ms":66222,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62L05","62F12","60F05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimism—implemented as variance inflation or a mean bonus—makes Thompson sampling stable and restores the validity of standard Wald confidence intervals under adaptive data collection.","keywords":["Thompson sampling","adaptive inference","stability","multi-armed bandits","optimism","variance inflation","Wald confidence intervals","posterior sampling"],"falsifier":"Simulate variance-inflated TS for K=3 with means (0,0,-Delta) and a growing horizon, and check whether N_{a,T}/(sigma(A) log T) converges to 2/Delta^2 for the suboptimal arm while both optimal arms receive N_{a,T}/T converging to 1/2; a persistent divergence at large T, or a third-arm counterexample to the cited suboptimal-pull bound, would refute the stability claim.","tokens_in":28187,"feed_emoji":"🎰","tokens_out":5623,"duration_ms":58050,"temperature":0.7,"pith_summary":"The paper sets out to show that a small dose of optimism repairs a known failure of Thompson sampling: under adaptive data collection, vanilla TS can produce pull counts that wander randomly, so the usual t-test and confidence intervals built from sample means are invalid. It proves that two optimistic variants—one that inflates the posterior sampling variance, one that shifts the posterior mean by a bonus—make each arm's pull count concentrate around a deterministic sequence, for any number of arms and even when several arms are tied for best. When pull counts concentrate, studentized sample means become asymptotically normal and standard Wald intervals recover their nominal coverage. The cost is a modest, slowly growing increase in regret, which the paper presents as a small price of stability. This matters for online experiments and adaptive trials, where analysts want both low regret and trustworthy p-values.","feed_headline":"Optimism stabilizes Thompson sampling for valid adaptive inference","feed_subtitle":"Inflating the posterior variance or adding a mean bonus restores classical Wald intervals under adaptive data collection.","key_machinery":"The stability argument rests on several interacting mechanisms. For variance inflation, the core object is a 'winner map' g(x): the probability that a pure-noise Gaussian index from arm i is the maximum when proportions are x. The map has a negative-feedback property—over-pulled arms win less often—which yields a dot-product inequality whose violation drives a quadratic Lyapunov function toward uniformity; a perturbation bound shows the actual Thompson selection probabilities track the ideal map up to a vanishing error. A separate rare-event analysis bounds the chance a suboptimal arm's index exceeds the best optimal-arm index, giving sharp exponential selection probabilities that are sandwi","core_discovery":"On the paper's own terms: with Gaussian unit-variance rewards and K fixed, if Thompson sampling draws indices from a posterior whose variance is inflated by a factor sigma(A) growing slower than T/log^2 T, or from a pseudo-posterior whose mean is shifted by sqrt(2 beta(A) log T / N_{a,t}) with beta(A) growing slowly, then N_{a,T}/N*_{a,T} converges in probability to 1, where the deterministic scale is T/m for optimal arms and 2 c(A) log T / Delta_a^2 for suboptimal arms. Consequently studentized sample means converge to N(0,1) and Wald confidence intervals for each arm mean have asymptotic coverage 1-alpha. The paper frames optimism as the unifying mechanism: either form of upward bias preve","pith_inferences":["If optimism is the stabilizing mechanism, other optimistic posterior rules—such as upper-quantile indexing or drawing multiple posterior samples and taking their maximum—should also stabilize TS; the mean-bonus proof already points in that direction.","A concrete boundary test: replace Gaussian noise with a non-Gaussian sub-Gaussian distribution. The proofs rely on Gaussian posterior updates and Gaussian tail bounds, so the current guarantee is for Gaussian rewards despite the abstract's wider promise; a sub-Gaussian extension would require new concentration tools.","The winner-map negative feedback resembles balancing mechanisms in other randomized allocation problems; the same Lyapunov argument may provide stable inference for a broader class of randomized rules whose selection probabilities decrease in their own allocation share.","For practitioners, the result suggests a simple recipe—run optimistic TS and use ordinary sample-mean confidence intervals without debiasing—but the slowly growing regret inflation should be weighed against the convenience of classical inference in a specific experiment."],"forward_implications":["Variance-inflated TS is stable for any K >= 2, including multiple optimal arms, resolving the K-armed extension left open in prior two-armed work.","Mean-bonus TS—a randomized analogue of optimistic bonus rules—is stable with the same sharp suboptimal pull asymptotics, isolating optimism as the key driver.","Under either variant, studentized sample means are asymptotically normal and Wald confidence intervals have nominal coverage despite adaptive sampling.","Suboptimal arms are pulled on a logarithmic scale: 2 sigma(A) log T / Delta_a^2 under variance inflation and 2 beta(A) log T / Delta_a^2 under a mean bonus.","Regret grows by only a mild slowly varying factor (e.g., (log log T)^2) relative to vanilla Thompson sampling."],"fun_headline_variants":["Optimism makes Thompson sampling statistically stable","Valid adaptive inference via optimistic Thompson sampling","Optimism restores classical inference in Thompson sampling","Thompson sampling stabilized by optimism for valid tests","Optimistic TS yields valid Wald intervals in bandits"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The proof borrows a prior bound on the expected number of suboptimal pulls, developed for two-armed TS, and applies it to the K-armed setting without proving it there; it also assumes Gaussian unit-variance rewards, even though the abstract advertises sub-Gaussian noises, so the argument stands or falls on those two premises.","fun_headline_variants_meta":{"raw":{"variants":["Optimism makes Thompson sampling statistically stable","Valid adaptive inference via optimistic Thompson sampling","Optimism restores classical inference in Thompson sampling","Thompson sampling stabilized by optimism for valid tests","Optimistic TS yields valid Wald intervals in bandits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1136,"prompt_tokens":805,"completion_tokens":331,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":264}},"tokens_in":549,"tokens_out":331,"duration_ms":3946,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:02:18.918358+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate variance-inflated TS for K=3 with means (0,0,-Delta) and a growing horizon, and check whether N_{a,T}/(sigma(A) log T) converges to 2/Delta^2 for the suboptimal arm while both optimal arms receive N_{a,T}/T converging to 1/2; a persistent divergence at large T, or a third-arm counterexample to the cited suboptimal-pull bound, would refute the stability claim.","supporting_citations":[],"review_version":1}