{"id":"051a0786-42b2-4b43-9c96-be7294ea8206","arxiv_id":"2505.14458","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An adaptive histogram estimator with a data-driven penalty achieves oracle risk bounds for transition densities of controlled Markov chains with continuous states and actions, without smoothness or control-distribution assumptions.","lead":"Researchers propose a method for estimating the transition rule of a controlled Markov chain, the probability law that maps the current state and an applied action to the next state, without knowing in advance how smooth that law is or how the actions were chosen.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proof of Theorem 1 misapplies Proposition 10: the union bound in §B.2 needs to control H2(s_hat_m, f), but Proposition 10 bounds H2(s, f2); the central oracle inequality is not established as written.","rationale":"The paper's headline contribution is Theorem 1, which claims an adaptive oracle inequality for transition densities of controlled Markov chains with no assumptions on the controls. If this theorem is unproven, the main advertised result collapses, and the deterministic Theorems 2 and 3 also lose their support because their proofs use Theorem 1. The reader's weakest_assumption focused on the exponential strong-mixing condition in the deterministic-Hellinger theorems; that is a legitimate concern, but it is not the most load-bearing issue. The more fundamental problem lies inside the proof of Theorem 1 itself: Proposition 2's union bound attempts to control a supremum involving H2(s_hat_m, f), while Proposition 10, as stated and proved, gives concentration for H2(s, f2). These are different objects, and no supplied inequality bridges them. This is not merely a missing detail; it is an invalid step in the central proof. The proposed concrete test—re-deriving the union bound from Proposition 10—would settle whether the gap is real or whether a corrected version of Proposition 10 with H2(f1, f2) can be proved. Since the proof as written does not establish Theorem 1, the appropriate verdict is REJECT: the central claim is unsupported. I therefore partially agree with the reader's assessment (the mixing concern is valid) but disagree that the weakest assumption is in the deterministic theorems; the proof gap in Theorem 1 is more fundamental. No code, data, or formal verification is provided to independently support the main result, so the proof gap is decisive.","tokens_in":51681,"tokens_out":22047,"duration_ms":183711,"concrete_test":"Re-derive the step after Eq. (B.13) by substituting f1 = s_hat_m and f2 = f into Proposition 10, and check whether the resulting inequality controls H2(s_hat_m, f) or H2(s, f). Then determine whether Proposition 10 can be restated with H2(f1, f2) on the left-hand side using the current definition of T and Lemma 16; if the needed inequality is not derivable from Lemma 17, the gap is confirmed. An independent check is to verify the proof of Proposition 2 by tracing each union-bound step and showing explicitly how the probability bound for the supremum involving H2(s_hat_m, f) follows from Proposition 10 as stated.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is Theorem 1, an oracle inequality for the selected histogram under empirical Hellinger loss with no assumptions on the control sequence. Its proof rests on Proposition 2 (Section B.2). In Case II of that proof, after bounding H2(s_hat_m, s_hat_mhat) by gamma(m)+1/n, the argument must control the probability that the supremum over f in s_m' of [ (3/4)(1-1/sqrt2) H2(s_hat_m, f) + T(s_hat_m, f) - pen(m') ] is large (Eq. B.13). The paper asserts this follows from Proposition 10 by a union bound. However, Proposition 10 (stated in Appendix A and proved in B.10) bounds the event with (3/4)(1-1/sqrt2) H2(s, f2) + T(f1, f2) on the left-hand side, not H2(s_hat_m, f). The quantity H2(s_hat_m, f) is the empirical Hellinger distance between two random histograms and depends on the sample; no inequality in the paper relates it to H2(s, f2) in a way that makes the union bound valid. Consequently, the proof of Proposition 2, and hence Theorem 1, has a gap. The deterministic Theorems 2 and 3 invoke Theorem 1 in their proofs, so the gap propagates. This is a load-bearing correctness risk distinct from the exponential-mixing assumption identified by the reader.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a penalized histogram estimator for the transition density of a controlled Markov chain with compact continuous state and control spaces. The main theoretical object is Theorem 1, an oracle inequality under an empirical Hellinger loss that is asserted to hold without any assumptions on the distribution or dependence structure of the control sequence. Theorems 2 and 3 convert this into deterministic Hellinger risk bounds under exponential strong mixing (Assumption 1) and, in Theorem 3, under a finite expected return-time condition; Theorem 4 states a minimax lower bound under the same conditions. Applications to Hölder and Besov classes and to fully connected Markovian and non-Markovian controlled chains are also given.","tokens_in":51896,"tokens_out":23186,"duration_ms":208970,"significance":"If the proofs are correct, this would be a valuable contribution: an instance-dependent oracle inequality for non-stationary, non-ergodic controlled Markov chains, a deterministic-Hellinger extension using α-mixing rather than β-mixing, a Kac-type return-time lower bound (Lemma 25) of independent interest, and explicit minimax statements. The paper also provides useful bridge results to existing adaptive density estimation theory and to fully connected models. However, the central proof of Theorem 1 contains a specific gap in the union-bound step, and the proofs of Theorem 2 and Theorem 4 contain direction errors. These are load-bearing issues, not presentation defects, so the current version cannot be accepted as written.","major_comments":[{"comment":"The proof of Proposition 2, and hence of Theorem 1, is missing a valid step at the union bound. The deviation event that must be controlled is of the form sup_{f∈s_{m'}, m'∈M_l} [(3/4)(1−1/√2)H2(ˆs_m,f)+T(ˆs_m,f)−pen(m')] + pen(m) + 1/n ≥ RHS, whose leading Hellinger term is H2(ˆs_m,f), the empirical Hellinger distance between two data-dependent histograms. Proposition 10, however, bounds an event whose leading term is (3/4)(1−1/√2)H2(s,f2) with f1,f2 piecewise constant functions and with the true transition density s in the Hellinger term. No inequality in Section B.10 or in Appendix A relates H2(ˆs_m,f) to H2(s,f2) in a way that makes the proposed union bound valid. In addition, Eq. (B.13) as printed bounds P(sup X_f ≤ RHS) by a sum of probabilities P(X_f ≤ RHS), which is the reverse of a tail union bound; even after correcting '≤' to '≥', the mismatch between H2(ˆs_m,f) and H2(s,f2) remains. Since Theorem 1 rests on Proposition 2, the central oracle inequality is not established as written.","section":"Section B.2, Eq. (B.13) and Proposition 10"},{"comment":"The transition from Proposition 23 to the displayed 'only need to upper bound R(n)' formula is invalid in direction. Proposition 23 gives a remainder with denominator 4C∆ρ⋆(Sr), while the proof replaces this by the smaller denominator 4C∆ sup_i P((Xi,ai)∈Sr). For fixed numerator A>0, the function d ↦ exp(−A/d) is increasing in d>0, so decreasing the denominator makes the remainder term smaller, not larger. The proof therefore establishes an upper bound only for a quantity that is no larger than the R(n) that actually needs to be controlled in Proposition 23 and in the statement of Theorem 2. The later replacement of sup_i P((Xi,ai)∈Smin) by P((Xi,ai)∈Smin) has the same direction problem. The proof must keep ρ⋆, or an upper bound on it, throughout the derivation.","section":"Section B.19, proof of Theorem 2"},{"comment":"The minimax statement in Eq. (3.5) has the wrong inequality sign. Part 2 of Proposition 19 proves that there exists a controlled Markov chain for which no estimator satisfies E[h2_n(s,ˆs)] ≤ 1/(2(1+π^2)); this is a lower bound of the form inf_ˆs sup_s E[h2_n(s,ˆs)] ≥ 1/(2(1+π^2)), not the upper bound '≤' printed in Eq. (3.5). The proof is internally inconsistent with the statement. Additionally, the derivation of Proposition 19's lower bound shows P(h2_n(s,ˆs)>ε^2) > c for ε∈(0,1/32); integrating over t=ε^2 introduces a factor of 1/1024 that is not reflected in the constant 1/(2(1+π^2)). Both the direction error and the missing integration factor affect the paper's main minimax-optimality claim and need to be corrected.","section":"Section 3.3, Theorem 4 and Proposition 19"}],"minor_comments":[{"comment":"There is a typo: 'Theroem 4' should read 'Theorem 4'.","section":"Section 1, Technical Contributions"},{"comment":"In the Case I display, H2(ˆsm, ˆsm) appears on the right-hand side; as printed this term vanishes, making the inequality trivial. The intended quantity appears to be H2(s, ˆsm) or an analogous term coming from Proposition 10.","section":"Section B.2, Case I"},{"comment":"The denominator notation '4n−1' in the statement of Theorem 2 is ambiguous; it should read 4n^{−1}, as in the proof.","section":"Section B.19, Theorem 2"},{"comment":"The sentence 'The other case is handled similarly with more careful book-keeping' is too terse for a result that relies on identifying ρ⋆; the missing case should be written out or suppressed by a uniform argument.","section":"Section 3.2, proof of Corollary 1"},{"comment":"The text says 'R(1)(n) = o(R(1)(n))'; this should be 'R(1)(n) = o(R(2)(n))'.","section":"Section B.5, proof of Proposition 5"},{"comment":"The abstract contains an awkward inserted '{and}' in 'minimizes a loss function {and} fitting the observed data well'; this should be cleaned up.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and I see no citation or novelty concern. The issues are technical: the central oracle-inequality proof has a real gap, and Theorem 2 and Theorem 4 have direction errors. These are fixable in principle, but they require substantial reworking of the proofs rather than cosmetic changes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read 2505.14458 carefully, including the appendices. The short version: the paper tackles a real problem and sets up a sensible framework, but the proof of the main result has a genuine gap, and a secondary result has a sign error. This should go to a serious referee, but it is not ready as-is.\n\nWhat is new: first adaptive estimator for transition densities of controlled Markov chains with continuous state/action and arbitrary control sequences. The chosen estimator (penalized dyadic histograms via a contrast) is natural, and the ambition—oracle bounds under empirical Hellinger with no assumptions on the controls—is worthwhile. The appendix contains some independent nuggets: the Kac-type lower bound (Lemma 25) and the mixing lemma for fully connected chains (Lemma 6) look plausible and could be useful elsewhere.\n\nSoft spots, in proportion:\n\n1. The proof of Theorem 1 misapplies Proposition 10. Proposition 10 bounds a probability involving H2(s, f2) and T(f1, f2) for fixed piecewise-constant f1, f2. In the proof of Proposition 2 (Section B.2, displayed eq. B.13), the argument needs to control H2(ŝ_m, f) and T(ŝ_m, f), where ŝ_m is the random histogram. No inequality in the paper relates H2(ŝ_m, f) to H2(s, f2) in a way that makes the union bound valid. The sketch in Appendix A does not address this; it just says the step follows. Since Theorem 1 is the paper's central contribution, and Theorems 2 and 3 invoke it, this gap is load-bearing. I don't see an easy fix in the text as written.\n\n2. Theorem 4's eq. (3.5) states inf sup E[h2_n] ≤ 1/(2(1+π^2)), while the proof (B.13) establishes a lower bound. The inequality is backwards.\n\n3. Lesser issues: Corollaries 2 and 3 are sketched by reference to [10]; the universal constants are existential; no numerical experiments. These are addressable.\n\nOn mixing: the paper honestly acknowledges that the deterministic-Hellinger results require exponential α-mixing (Assumption 1), with no polynomial analogue. That is a limitation, not a flaw, and the authors say so.\n\nVerdict: the framework and some technical pieces have value, but the main theorem is not established as written. A referee could reasonably ask for a corrected proof of Theorem 1 before accepting. I would send it out rather than desk-reject, because the problem matters and the approach is credible; the gap may be fixable.","headline":"The framework is right, but the proof of Theorem 1 misapplies Proposition 10 and the main oracle inequality is not established as written; the paper deserves a serious referee, not desk rejection.","tokens_in":52570,"tokens_out":4787,"would_cite":false,"duration_ms":40813,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-07T15:34:46.343717+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}