{"id":"0f67f7b1-3ca8-4fec-831b-ed3e2fbdc27c","arxiv_id":"2607.16407","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Single-hidden-layer neural-network OWL is extended to sparse and kernel variants, with excess-risk rates, global-convergence and implicit-bias results.","lead":"This paper develops theory and algorithms for neural-network outcome weighted learning with ℓ1-regularized variable selection and kernel-based approximations, plus mean-field results on gradient descent. It gives convergence rates and an implicit-bias characterization that could inform precision-medicine treatment rules from trial data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Optimization results are formally proven only for a smooth signed-square reparameterization of ReLU, not for the actual nonsmooth ReLU network optimized in the paper; no transfer theorem closes this gap.","rationale":"The reader's weakest assumption identifies the same gap: the formal optimization theorems are for a smooth signed-square reparameterization, while the main text and algorithms use ReLU directly. I agree this is the single most load-bearing concern. The statistical-rate portion is largely self-contained and plausible, and the supplement contains substantial technical work, but the optimization claims are the least secure. The fix is not trivial because the signed-square model changes both the parameter space and the gradient-flow geometry; a transfer theorem is needed. A computational experiment alone might not settle the issue, so the proposed test is analytical: verify whether the two flows coincide under the natural bijection, or whether a quantitative coupling exists. If the transfer fails, the paper's central optimization claims would need to be explicitly scoped to the reparameterized model, which would weaken but not necessarily invalidate the whole paper. The existing conditional verdict remains appropriate; I would not move to reject, because the statistical results and much of the supplement are valuable and the optimization gap may be repairable.","tokens_in":56163,"tokens_out":14448,"duration_ms":127761,"concrete_test":"Independently derive the one-particle gradient-flow ODE for the direct ReLU parameterization θ=(ζ,w,b), Φ(θ)(x)=ζ(w^T x+b)_+, and for the signed-square feature map of Proposition S2.7 under the natural bijection τ ↦ (sign(ζ), sign(w)√|w|, sign(b)√|b|). Check whether the two flows are conjugate via a time-preserving transformation that also preserves the limiting variation-norm max-margin problem (11). If no such conjugacy or quantitative approximation theorem is supplied, the formal results should be restated as applying to the signed-square model, and Propositions 3.13–3.14 should be weakened or explicitly conditioned on the reparameterized model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's third advertised contribution is global convergence and implicit bias for gradient-descent-trained ReLU NNOWL (Section 3.6, Propositions 3.13–3.14). The formal supplement proves these for a smooth signed-square reparameterization on two copies of R^{d+1}: Proposition S2.7 constructs the feature map Φ(τ)(x)=±σ(Σ s(τ_k)x_k+s(τ_{d+1})) with s(t)=t|t|, and Theorems S2.15–S2.17 rely on Assumptions S2.14(i)–(iii), which require Φ(·,x_i) to be differentiable with Lipschitz-continuous gradient and subanalytic. The standard ReLU parameterization Φ(ζ,w,b)=ζ(w·x+b)_+ used in the algorithmic sections and experiments is not differentiable at the kink, so it does not satisfy these hypotheses. No theorem in the paper shows that the Wasserstein gradient flow of the signed-square model is equivalent to, or even approximates, the subgradient flow of the direct ReLU model, despite the natural bijection between the representations. Because the reparameterization changes the particle dynamics and the norm geometry, equivalence is not automatic. Consequently, the main-text propositions overstate what has been established: they are proved for a smoothed surrogate model, not for the ReLU networks actually trained. This is load-bearing because the optimization theory is a central claimed contribution and because the statistical-rate results alone do not cover the nonconvex optimization behavior of the implemented estimators.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends outcome weighted learning (OWL) to single-hidden-layer neural networks under three regularization schemes: ridge (NNOWL-Ridge), ℓ1 input-weight regularization for variable selection (NNOWL-VS), and an induced kernel method (NNOWL-Kernel), with a residual-weighted analogue (NNRWL). The main theoretical claims are non-asymptotic excess surrogate-risk bounds: for a Sobolev target in H^t, the ridge and kernel estimators achieve O(n^{-t/(d+5)}) when t < d/2+5/2 and parametric n^{-1/2} otherwise, with dimension-adaptive rates in latent-subspace and coordinate-sparse settings for ℓ1 regularization. The paper also claims global convergence and implicit margin-maximization for gradient-descent-trained wide ReLU networks under the OWL surrogate loss, obtained via mean-field theory. Simulations and an Alzheimer's disease application illustrate the methods.","tokens_in":56578,"tokens_out":4048,"duration_ms":39344,"significance":"If the statistical rates are correct, the paper provides useful generalization guarantees for a practical class of neural-network ITR estimators, and the kernel/VS variants are a natural extension of existing OWL methodology. The paper is also honest in several places about the informal nature of the optimization statements, and the supplement contains detailed Rademacher and mean-field arguments. However, the optimization contribution — advertised as a central part of the paper — is currently not proven for the actual ReLU networks used in the algorithms and experiments, and the statistical rates rest on an unproved external lemma. These issues materially affect the strength of the advertised contributions.","major_comments":[{"comment":"Propositions 3.13 and 3.14 are stated for ReLU activation, but the formal results in the supplement are proved for a smooth signed-square reparameterization of ReLU on two copies of R^{d+1}. Assumption S2.14(iii) requires Φ(·,x_i) to be differentiable with Lipschitz-continuous gradient and subanalytic; the standard ReLU parameterization Φ(ζ,w,b)=ζ(wᵀx+b)_+ is not differentiable at the kink. The paper provides no transfer theorem showing that the Wasserstein gradient flow of the signed-square model is equivalent to, or approximates, the subgradient flow of the direct ReLU model. This gap is load-bearing because the optimization results are a central advertised contribution and because the statistical rates alone do not cover the nonconvex optimization behavior of the implemented estimators.","section":"§3.6; Supplement §2.2–2.3 (Assumptions S2.14, Proposition S2.7, Theorems S2.15–S2.17)"},{"comment":"Lemma 3.3, the Sobolev-to-variation approximation bound, is imported from Wang (2026, Lemma 3.1) without proof or even a full statement. This lemma is used to derive every non-asymptotic rate in Propositions 3.6, 3.10, and 3.12, so it is not a peripheral ingredient. Since Wang (2026) is a preprint, the manuscript should either reproduce the proof in the supplement or state the lemma completely and precisely. As written, the reader cannot verify the claimed rates.","section":"§3.2, Lemma 3.3"},{"comment":"The global-convergence results are conditional on the Wasserstein gradient flow converging to some distribution. The theorems state that if the flow converges, then the limit is a global minimizer; they do not establish convergence of the flow in the first place. The main-text wording 'global convergence of gradient descent' therefore overstates the formal content. The paper should either prove convergence of the flow under the stated hypotheses or explicitly qualify the results as convergence-conditional.","section":"§3.6, Propositions 3.13–3.14; Supplement Theorems S2.6, S2.9, S2.12"},{"comment":"The proximal-gradient algorithm for NNOWL-VS is justified by asserting that the loss term L is differentiable in w with Lipschitz-continuous gradient. For a ReLU network, L is not differentiable at the kinks, and the paper does not analyze the subgradient or proximal subgradient dynamics. Moreover, the rate results in Theorem 3.8 and Proposition 3.10 concern empirical risk minimizers over the constrained class Θ(1), not the iterates of the proposed algorithm. The connection between the computational procedure and the theoretical class is not established.","section":"§2.2 and §3.4 (NNOWL-VS)"}],"minor_comments":[{"comment":"The keyword 'global convergency' should be 'global convergence'.","section":"Keywords and typing"},{"comment":"The parenthetical 'this needs explanations (that kernel methods do not involve gradient decent iterations)' appears to be an editorial note left in the text and should be removed or rewritten.","section":"§4.2 (editorial note)"},{"comment":"The supplement table of contents contains unresolved cross-references such as 'Proof of Lemma??' and 'Proof of Theorem??'. These should be replaced with actual numbering.","section":"Supplement TOC"},{"comment":"The main text cites Chaintron & Diez (2022a,b) and Chizat & Bach (2018) only in the supplement; the main reference list does not include all supplement citations, and the Chaintron & Diez entries lack full publisher/page details in some places.","section":"References"},{"comment":"Some notation is introduced with double occurrence of definitions (e.g., R and Rℓ in §3 and again in the supplement); a consolidated notation table or cross-reference would improve readability.","section":"General presentation"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid statistical-rate core, but the optimization claims are substantially overstated relative to the formal supplement. The smooth-reparameterization gap is not a mere technicality: it affects the exact object being optimized in the main text and experiments. I would require either a genuine convergence proof for nonsmooth ReLU mean-field dynamics or a clear rewriting of the main-text propositions as results for the signed-square model. The reliance on Lemma 3.3 from a self-cited preprint is also a concern that should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: the paper genuinely adds rates for ℓ1-regularized and kernel random-feature NNOWL, and the weighted mean-field analysis is a real extension — but the optimization theorems are only proved for a smooth signed-square reparameterization of ReLU, not the network you actually train, and no transfer result closes that gap.\n\nWhat's new and good: NNOWL-VS and NNOWL-Kernel estimators are new. The rate proofs adapt Bach (2024) and Chizat & Bach (2018, 2020) with weights inserted; the supplement is detailed and the Rademacher/calibration machinery is standard and does what it needs to. The rates — n^{-t/(d+5)} with log factors for the ℓ1 version — are a legitimate extension. The simulations are honest, show the methods are competitive, and report computing times in the supplement. The statistical-rate half of the paper is in good shape.\n\nSoft spots, in proportion. First and most important: Propositions 3.13 and 3.14 in the main text say \"ReLU activation\" and are informal. The supplement proves global convergence and implicit bias only for the smooth reparameterization Φ(τ)(x) = ±σ(Σ s(τ_k)x_k + s(τ_{d+1})) with s(t)=t|t|; it requires differentiability and subanalyticity assumptions that the standard ReLU parametrization fails. The paper does not show that the gradient flow of the smoothed model limits to or approximates the subgradient flow of the actual ReLU network. That is a real gap in a central claimed contribution. The rates stand without it, though.\n\nSecond, the calibration transform and Sobolev approximation lemma come from the author's own Wang (2026). Not circular, but it makes part of the paper dependent on a companion manuscript.\n\nThird, the simulation comparison against Jiang et al. (2024) uses a 720/80 train/tuning split rather than their 10-fold CV; calling that \"comparable\" is a stretch. Also, no code is released yet, only a promise. Minor but easy to fix.\n\nWho it's for: anyone working on theory for neural-network ITR estimators. If a colleague referees it, the rates part deserves a proper look; the optimization section needs revision — either prove a transfer result or restate the claims as being for the smooth model. I'd send it to review with a request for major revision on that point.","headline":"Rates for ℓ1/kernel NNOWL are solid, but the optimization claims are proved only for a smooth signed-square surrogate, not for the ReLU networks actually trained — a real gap in a central contribution.","tokens_in":56986,"tokens_out":2816,"would_cite":true,"duration_ms":28230,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that single-hidden-layer neural networks trained with outcome-weighted learning—recasting treatment assignment as a weighted classification problem—carry provable non-asymptotic convergence rates, and that gradient descent","keywords":["outcome weighted learning","individualized treatment rules","neural networks","convergence rates","variable selection","kernel methods","implicit bias","mean-field theory"],"falsifier":"Train a wide one-hidden-layer ReLU network on a separable OWL problem with logistic loss and no explicit regularization; if the normalized margin of the limiting predictor fails to approach the variation-norm max-margin value computed from the same training points, the implicit-bias claim would be refuted. Equivalently, finding a finite-width run that converges to a separator with demonstrably larger variation norm than another achievable separator would contradict the claimed bias.","tokens_in":56071,"feed_emoji":"🧠","tokens_out":4325,"duration_ms":44350,"temperature":0.7,"pith_summary":"Individualized treatment rules assign treatments based on patient covariates to maximize expected clinical outcomes, and outcome weighted learning (OWL) does this by turning treatment assignment into a weighted classification problem. This paper extends OWL to neural networks with ridge, L1 variable-selection, and kernel-style regularizers, and claims explicit non-asymptotic bounds on expected excess risk for all three. The rates adapt to intrinsic structure: a target that is smooth in a low-dimensional subspace or depends on only a sparse set of covariates is learned at a rate set by that intrinsic dimension, not the ambient one. The paper also claims that, in the infinite-width limit, gradient descent converges to a global optimum and, without explicit regularization, is biased toward low-variation max-margin treatment rules. These results matter because they connect a practically flexible estimator to the kind of statistical and optimization guarantees that make precision-medicine recommendations trustworthy.","feed_headline":"Rates proven for neural-network personalized treatment rules","feed_subtitle":"New bounds show learning adapts to sparsity and latent dimension, plus a max-margin bias for gradient descent.","key_machinery":"The load-bearing object is the variation norm V1(f): the minimum total variation of a signed measure over a compact dictionary of normalized ReLU neurons that represents f. For finite-width networks this norm is the infinite-width analogue of an L1 penalty on output weights, and it is what connects smoothness of the target to approximation error. The rate proofs combine a calibration transform that converts excess surrogate risk into excess 0–1 risk, Rademacher-complexity bounds for the weighted loss class, and a balancing lemma that optimizes the radius of the variation-norm ball. The optimization results use a mean-field formulation in which the m hidden neurons are interacting particles a","core_discovery":"The central claim is that the expected excess surrogate risk of the neural-network OWL estimator is governed by an approximation–estimation tradeoff centered on the variation norm of the target function—the minimal total variation of a signed measure over normalized ReLU neurons representing the function. For a target in a Sobolev smoothness class of order t, the paper derives a rate of n^{-t/(d+5)} when t is below a dimension-dependent threshold and the parametric n^{-1/2} rate when t is large; the L1-penalized variant pays only a log(d) factor but replaces the ambient dimension with the number of active covariates, and the kernel variant attains the same nonparametric exponent using a Hilb","pith_inferences":["A testable extension left implicit by the paper: the implicit-bias characterization suggests that the normalized margins of wide ReLU networks on separable OWL problems converge to the variation-norm max-margin value; this can be checked empirically and would confirm whether the outcome weights affect only the trajectory, not the limiting separator class.","If the approximation–estimation balancing argument extends to deeper networks or to multi-stage dynamic treatment regimes, the same variation-norm machinery could provide a general template for rate guarantees in personalized medicine beyond single-hidden-layer models.","The calibration-transform result implies that practitioners can rank candidate treatment-rule estimators by their excess surrogate risk rather than relying solely on noisy value-function estimates, a practical consequence the paper does not develop.","The theory treats the propensity score as known or reliably estimated; in observational settings where propensity estimation is itself difficult, the stated rates would need to account for an additional estimation error term that is not addressed here."],"forward_implications":["When the optimal treatment rule is sufficiently smooth, NNOWL achieves the parametric n^{-1/2} rate, so sample complexity is driven by the number of parameters, not the network width.","When the treatment effect depends on a low-dimensional latent subspace or a sparse set of covariates, the convergence-rate exponent improves accordingly, making the curse of dimensionality less severe.","The L1 variable-selection variant can set input weights exactly to zero through soft-thresholding, giving a concrete mechanism for nonlinear variable selection in treatment-rule learning.","The implicit-bias result implies that even without explicit regularization, wide unpenalized networks trained on separable OWL problems will converge to a separator with minimal variation norm, tying regularization to a geometric quantity.","The kernel-induced NNOWL variant reproduces the same nonparametric rate as the fully trained network while replacing nonconvex optimization with convex RKHS estimation, offering a computationally simpler route in practice."],"fun_headline_variants":["Rates proven for neural-network treatment rules","Neural network rules adapt to sparsity and dimension","Gradient descent bias found in NN treatment rules","Kernel NN matches nonparametric rates for ITRs","New bounds for individualized treatment rule learning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The global-convergence and implicit-bias results are proved for a smooth signed-square reparameterization of ReLU in the supplement, and the paper does not establish that these guarantees transfer to the non-differentiable ReLU network actually used in the algorithms and experiments.","fun_headline_variants_meta":{"raw":{"variants":["Rates proven for neural-network treatment rules","Neural network rules adapt to sparsity and dimension","Gradient descent bias found in NN treatment rules","Kernel NN matches nonparametric rates for ITRs","New bounds for individualized treatment rule learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000143,"raw_usage":{"total_tokens":1024,"prompt_tokens":774,"completion_tokens":250,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":180}},"tokens_in":518,"tokens_out":250,"duration_ms":2845,"temperature":1.0,"reasoning_tokens":180,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T21:03:36.156725+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a wide one-hidden-layer ReLU network on a separable OWL problem with logistic loss and no explicit regularization; if the normalized margin of the limiting predictor fails to approach the variation-norm max-margin value computed from the same training points, the implicit-bias claim would be refuted. Equivalently, finding a finite-width run that converges to a separator with demonstrably larger variation norm than another achievable separator would contradict the claimed bias.","supporting_citations":[],"review_version":1}