{"id":"800d8553-5b47-478e-bfcc-61963532da8e","arxiv_id":"1908.05531","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"For exponential two-armed bandits, the limiting Bayesian regret equation matches the Gaussian equation, implying batch processing is asymptotically as efficient as one-by-one control.","lead":"The paper shows that when rewards follow exponential distributions, the optimal two-armed bandit strategy has the same large-horizon limit as the Gaussian case. A generalist reader may care because this suggests batched data processing loses no asymptotic efficiency for exponential rewards.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central equivalence rests on an unproved limit interchange: the Taylor/CLT expansion that turns the discrete recursion (4.9) into the PDE (4.15) assumes uniform smoothness of the normalized risk that is not established; a numerical convergence check would settle whether the claimed…","rationale":"The recursive unnormalized equation (3.3) is exact and the moment identities (4.5) are algebraically consistent; I see no error in the exponential-family manipulation that removes the prior from the transition kernel. The reader's weakest assumption is the same one I would defend: the passage from (4.9) to (4.15) is a formal asymptotic expansion with no control of the remainder. The normalized risk r_tilde is defined only on the discrete grid and, as the value function of a min-Bellman recursion, it can have kinks at the switching boundary; the second derivative in (4.11) need not exist uniformly in N, and the CLT error in (4.2) is not shown to vanish after integration against the Gamma kernel. Since the central conclusion ('same description as Gaussian') is exactly the assertion that this limit PDE is correct, this gap is load-bearing. I also note that (5.4) as typeset has εx1 in the Gaussian kernel; a direct moment calculation gives mean εx1/t1, so the printed equation would not produce the drift term in (4.15). This looks like a typo for δx1, but it reinforces that the comparison step needs to be checked. The proposed numerical test would provide evidence either way; if the discrete risk converges to the PDE value, the formal limit is justified in practice and the verdict can be upgraded.","tokens_in":7555,"tokens_out":19137,"duration_ms":175691,"concrete_test":"Solve the normalized discrete recursion (3.3)/(4.9) on a fine grid for a fixed smooth, compactly supported prior rho, for N = 100, 400, 1600, 6400 (epsilon = 1/N), and compute the normalized Bayesian risk from (4.10). Compare with a numerical viscosity solution of the PDE (4.15)-(4.16) on the same domain. If the difference does not tend to zero (ideally as O(epsilon)), the Taylor/CLT limit interchange producing (4.15) is not justified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that exponential and Gaussian bandits share the limiting PDE (4.15). The derivation of (4.15) from the discrete Bellman recursion (4.9) is formal: (4.2) replaces the Gamma density f(X,n|m) by its CLT Gaussian form with a uniform o(1) error, and (4.11) expands the value function as r(x+δy,t+ε) = r(x,t+ε) + δ r'_x y + (1/2)ε r''_xx y^2 + o(ε). For this to be valid after integrating against the predictive kernel, r must be C^2 with second derivatives uniformly bounded in N, and the CLT error must remain o(ε) under the integral; neither is proved. In particular, the first-order drift term r'_x x_hat in (4.12) comes from the O(δ) mean of the Gamma kernel, so the o(ε) remainder in (4.11) has to be uniform in y over the unbounded integration range. The Gaussian comparison also needs attention: (5.4) as typeset has kernel argument εx1 - t1 y, whose first moment is εx1/t1 = O(ε); multiplied by δ this is O(ε^(3/2)), so the printed equation would not yield the drift term in (4.15). If this is a typo for δx1 - t1 y, the intended comparison works, but the text as written is inconsistent.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a Bayesian two-armed bandit with exponentially distributed rewards and finite horizon N. It sets up Bellman-type recursions for the risk, introduces a normalized version, and derives, in the regime where the two arm means are within O(N^{-1/2}), a limiting second-order PDE for the normalized risk. It then asserts that the Gaussian two-armed bandit from [5] leads to the same PDE and concludes that, asymptotically, batch processing does not enlarge Bayesian risk for exponential rewards.","tokens_in":7895,"tokens_out":10210,"duration_ms":92359,"significance":"If the derivation were rigorous, the result would be a clean asymptotic equivalence between exponential and Gaussian bandits and would extend the batch-processing optimality from Gaussian to exponential rewards. The paper is also useful for presenting explicit recursive equations and a concrete scaling. However, the central equivalence is achieved through formal asymptotics: uniform CLT approximations and Taylor expansions with uniform error controls are asserted rather than proved, and the printed Gaussian equation in (5.4) is inconsistent with the claimed limit as written. The practical conclusion is therefore conditional on closing these gaps.","major_comments":[{"comment":"The replacement of the Gamma density by a Gaussian density with relative error o(1) and the resulting formula for G^(l) are used without uniform error bounds. Since the subsequent equalities (4.5) and the integrations in (4.9)-(4.12) involve unbounded functions of y and x, an error that is merely pointwise o(1) can become non-negligible after integration and after multiplication by delta and epsilon. The coefficients g^(l) in (4.3) are therefore not rigorously identified.","section":"Section 4, Eq. (4.2)"},{"comment":"The Taylor expansion of tilde r(x1+delta y, t1+epsilon, x2, t2) is legitimate for the stated conclusion only if tilde r is C^2 in x1 with second derivative uniformly bounded in (x,t,N) and the o(epsilon) remainder is uniform in y over the support of the kernel f1(1+y). None of these regularity properties is proved; the kernel has unit-order variance and unbounded support, so the remainder term can in principle contribute at order delta or epsilon. Equation (4.12) and hence the PDE (4.15) rest on this unproved limit interchange.","section":"Section 4, Eq. (4.11)"},{"comment":"The paper asserts without proof that the maximum expected losses take place in the domain |m1-m2| <= c N^{-1/2}. This domain restriction is the justification for the scaling (4.1) and for the entire limiting description; a reader cannot verify that the PDE characterizes the Bayesian or minimax risk of the original problem without a supporting argument or reference.","section":"Section 4, first paragraph"},{"comment":"As printed, the Gaussian recursion has kernel f_{t1(t1+epsilon)}(epsilon x1 - t1 y), which gives y mean epsilon x1/t1, so the first moment of the transition is O(epsilon); after multiplication by delta the drift contribution is O(epsilon^{3/2}) and cannot produce the term r'_x1 * x1/t1 in (4.15). The claimed identity of the exponential and Gaussian limiting PDEs therefore does not follow from the equation as written. If this is a typo for delta x1 - t1 y, with a correspondingly adjusted prefactor, the argument needs to be restated; the sentence \"one can verify\" is not a proof.","section":"Section 5, Eq. (5.4)"}],"minor_comments":[{"comment":"The notation in the denominator of (2.2) uses t1 and t2, while the function is defined with arguments n1 and n2; the notation should be made consistent.","section":"Section 2, Eq. (2.2)"},{"comment":"The double integral is typeset as \"integral integral infinity -infinity\"; it should be written as a double integral over R^2.","section":"Section 4, Eq. (4.10)"},{"comment":"The partial derivatives r'_t1, r'_x1, and r''_x1 x1 are used without being defined explicitly; a sentence identifying them as partial derivatives of tilde r with respect to the scaled variables would help.","section":"Section 4, Eqs. (4.12)-(4.13)"},{"comment":"The comparison with the Gaussian case relies entirely on the reference [5], but the paper does not state which theorem or equation from [5] is being transferred; adding a precise quotation would make the comparison checkable.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The central claim is plausible and the recursive setup is sound, but the asymptotic derivation is formal and the Gaussian comparison contains an apparent typo in the key equation. A revision needs to supply the missing uniform error estimates or explicitly state them as assumptions, and to correct or justify (5.4)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The new content is a recursive equation for exponential two-armed bandits, a scaling limit, and the observation that the limiting PDE matches the Gaussian case [5]. If correct, it extends [5]'s asymptotic framework to exponential rewards and gives theoretical cover for batched processing. The derivation from the Bellman recursion is standard, and the bookkeeping in Section 4 is mostly careful. I believe the authors are thinking about the right objects, and the equivalence claim is likely true.\n\nThe two main weaknesses are both at the limit interchange, as expected. Eq. (4.2) replaces the Gamma density with a Gaussian uniformly, and Eq. (4.11) does a second-order Taylor expansion with an o(epsilon) remainder; both have to survive integration against an unbounded kernel. That is asserted, not proved, and it is load-bearing for the PDE (4.15). The claim that maximum expected losses occur in the close-distributions domain is also unproved. Section 5 says 'one can verify' but gives no verification, and there is a concrete typo: Eq. (5.4) as typeset has kernel argument εx1 - t1 y, whose first moment is O(ε); multiplied by δ that is O(ε^(3/2)), so it cannot produce the drift term. The intended comparison likely needs δx1 - t1 y. The self-citation to [5] is fine; that is the right comparison, and the paper does not fit parameters.\n\nVerdict: the central conclusion is plausible, but the unproved limit interchange is serious enough that I would not treat the equivalence as established. A serious referee should ask for either a rigorous proof of the limit interchange or a convincing numerical convergence check. This is not a desk reject. It is a short, specialized paper for people working on asymptotic Bayesian bandits, and it deserves peer review with a request for revision.","headline":"Plausible and useful extension of Gaussian asymptotic analysis to exponential bandits, but the central limit interchange is unproved and one displayed equation has a typo.","tokens_in":8372,"tokens_out":2321,"would_cite":false,"duration_ms":24260,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["93E20","62L05","62C10","62C20","62F35"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes that, in the infinite-horizon limit, exponential two-armed bandits obey the same second-order PDE as Gaussian bandits, implying that batched processing is asymptotically as good as one-by-one play.","keywords":["exponential two-armed bandit","Bayesian risk","close distributions","second-order partial differential equation","Gaussian two-armed bandit","batch processing","recursive Bellman equation","regret"],"falsifier":"Compute the Bayesian risk for a concrete exponential two-armed bandit, say $m_1=m+\\Delta$ and $m_2=m-\\Delta$ with $\\Delta=O(N^{-1/2})$, by solving the recursion (2.3) for growing $N$, and compare the normalized risk with the Gaussian batch-processing value predicted by (4.15); if the normalized difference does not tend to zero, the claimed limit equivalence is wrong. Alternatively, check whether the residual in (4.12) induced by the $o(\\varepsilon)$ terms in (4.5) vanishes for a non-smooth value function.","tokens_in":7325,"feed_emoji":"🎰","tokens_out":4046,"duration_ms":37054,"temperature":0.7,"pith_summary":"This paper studies a two-armed bandit whose rewards are exponentially distributed with unknown means. It derives a recursive Bellman equation for the Bayesian risk and then passes to the limit of an infinite control horizon inside the close-distributions region. The limit is a second-order partial differential equation, and the paper shows this PDE is identical to the one obtained earlier for a Gaussian two-armed bandit. Because the Gaussian bandit is known to describe batched data processing, the paper concludes that for exponential rewards batching observations does not increase Bayesian risk asymptotically compared with optimal one-by-one processing.","feed_headline":"Exponential bandits reach the same limit as Gaussian bandits","feed_subtitle":"Batching rewards no longer raises Bayesian risk once the control horizon is large.","key_machinery":"The load-bearing object is the unnormalized Bayesian risk $\\tilde r(x_1,t_1,x_2,t_2)$ evolving under the recursive equation (4.9): each arm's continuation term is an integral of $\\tilde r$ against a Gamma-density kernel, rescaled with $\\varepsilon=N^{-1}$ and $\\delta=N^{-1/2}$. The argument works by replacing the Gamma density $f(X,n|m)$ with its Gaussian central-limit approximation, Taylor-expanding $\\tilde r$ in the increment $\\delta y$ and time step $\\varepsilon$, and collecting terms of order $\\varepsilon$ to obtain the PDE (4.15). The same PDE is then shown to follow from the Gaussian bandit's integro-difference equation, which establishes the claimed equivalence.","core_discovery":"The central claim is that as the horizon length $N$ tends to infinity, the normalized Bayesian risk of the exponential two-armed bandit satisfies the same second-order PDE (equation 4.15) as the Gaussian two-armed bandit, with the same initial condition. Consequently, the exponential and Gaussian bandits have identical asymptotically optimal strategies in the close-distributions domain, and the asymptotic value of the Bayesian risk is governed by the same constant. This equivalence allows the paper to conclude that batch processing, the practical situation modeled by Gaussian bandits, does not enlarge Bayesian risk for exponential rewards in the large-$N$ limit.","pith_inferences":["If the limit interchange is valid, a similar equivalence should hold for any one-parameter exponential-family reward whose sufficient statistics satisfy a central limit theorem, because only the first two moments appear in the scaling; the paper does not explicitly discuss this extension.","A testable extension is finite-$N$ simulation of the recursive equation (2.3) for exponential rewards; one would expect the difference between batched and one-by-one Bayesian risks to vanish as $N^{-1/2}$ or faster, but the paper reports no such numerics.","The PDE formulation implicitly predicts that the asymptotic optimal strategy depends on the prior only through the initial rescaled posterior $\\rho(v_1,v_2)$, a consequence of the scaling (4.1) that is not spelled out in the paper."],"forward_implications":["Bayesian strategies for exponential two-armed bandits can be approximated for large $N$ by solving the same PDE as for Gaussian bandits, so numerical methods developed for the Gaussian case carry over.","Batched processing of exponential rewards, collecting blocks before updating the strategy, does not asymptotically increase regret relative to fully sequential optimal control.","The asymptotic Bayesian risk scales as $(DN)^{1/2}$ times a constant determined by the PDE, where $D=m^2$ is the variance of an exponential reward.","The equivalence indicates that the close-distributions domain is the only region that determines the asymptotic risk, since that is where the PDE is posed."],"supporting_citations":[{"why":"Supplies the standard setting of two-armed bandit problems and regret, which the paper adapts to exponential rewards.","marker":"[1]"},{"why":"Provides the Bayesian sequential-control framework and recursive value-function equations that underlie (2.3)-(2.5).","marker":"[2]"},{"why":"Introduces the Gaussian two-armed bandit integro-difference equation and its batch-processing interpretation, which is the benchmark the paper matches in the limit.","marker":"[5]"}],"fun_headline_variants":["Exponential and Gaussian bandits share asymptotic risk limit","Batch processing no longer hikes Bayesian risk in exponential bandits","Exponential bandits match Gaussian asymptotics as horizon grows","Large-horizon exponential bandits match Gaussian risk exactly","Exponential bandit risk equals Gaussian at large horizon"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that replacing the Gamma density with a Gaussian approximation and Taylor-expanding the value function leave an error small enough to survive integration in the recursive equation, so that the limit equals the Gaussian bandit's PDE without extra terms.","fun_headline_variants_meta":{"raw":{"variants":["Exponential and Gaussian bandits share asymptotic risk limit","Batch processing no longer hikes Bayesian risk in exponential bandits","Exponential bandits match Gaussian asymptotics as horizon grows","Large-horizon exponential bandits match Gaussian risk exactly","Exponential bandit risk equals Gaussian at large horizon"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000547,"raw_usage":{"total_tokens":2529,"prompt_tokens":775,"completion_tokens":1754,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":391,"completion_tokens_details":{"reasoning_tokens":1675}},"tokens_in":391,"tokens_out":1754,"duration_ms":11185,"temperature":1.0,"reasoning_tokens":1675,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:10:52.011500+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the Bayesian risk for a concrete exponential two-armed bandit, say $m_1=m+\\Delta$ and $m_2=m-\\Delta$ with $\\Delta=O(N^{-1/2})$, by solving the recursion (2.3) for growing $N$, and compare the normalized risk with the Gaussian batch-processing value predicted by (4.15); if the normalized difference does not tend to zero, the claimed limit equivalence is wrong. Alternatively, check whether the residual in (4.12) induced by the $o(\\varepsilon)$ terms in (4.5) vanishes for a non-smooth value function.","supporting_citations":[{"cited_title":"Exponential two-armed bandit problem","cited_arxiv_id":"1908.05531","evidence_quote":"Supplies the standard setting of two-armed bandit problems and regret, which the paper adapts to exponential rewards."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Bayesian sequential-control framework and recursive value-function equations that underlie (2.3)-(2.5)."},{"cited_title":"close distributions","cited_arxiv_id":null,"evidence_quote":"Introduces the Gaussian two-armed bandit integro-difference equation and its batch-processing interpretation, which is the benchmark the paper matches in the limit."}],"review_version":1}