{"id":"d831110f-323b-4e4b-8e6d-af98331cbf28","arxiv_id":"2607.22467","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"For one-sided box-constrained quadratics, keeping the last k+1 projected-gradient iterates yields a Rademacher-complexity generalization bound controlled by the contraction factor and neighborhood radius.","lead":"This paper studies whether keeping the last few intermediate steps of a numerical solver, along with the final answer, can cheaply enlarge training data for models that suggest starting points for optimization problems. It derives a theoretical bound on how the number of kept steps and the solver's contraction rate affect generalization, with caveats in the proof.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3's proof treats the squared DDPM loss as if it were the norm loss, so Eq. (15)/(19) omit a factor A+√d (and B_l should be squared); the stated bound is not derived as written.","rationale":"The reader's weakest_assumption was the bounded-diffusion-noise assumption, which is a legitimate external-validity gap and is acknowledged by the authors as future work. However, I see an additional, more internal problem: the proof of Eq. (15) applies vector contraction to the norm map y↦||y-ε_i||, while the actual loss is the square of that norm. This is not a matter of extending the analysis to Gaussian noise; it is a mismatch between the function being bounded and the function whose Rademacher complexity is claimed. The same conflation appears in the definition of Bℓ, which is used as the bound on the squared loss in Theorem 2. These issues are repairable by inserting the correct Lipschitz constant and squaring the loss bound, and the qualitative conclusion about how k, ρ, r_k, γ, and N enter the bound would survive. Because the central theorem needs correction but not wholesale rejection, I do not move the reader's CONDITIONAL verdict; hence 'UNCHANGED'. My agreement is partial because the reader's stated weakest assumption is the bounded-noise condition, whereas I think the norm/square conflation is at least as load-bearing and located squarely inside the proof of Theorem 3.","tokens_in":6709,"tokens_out":8997,"duration_ms":101868,"concrete_test":"Independently re-derive R̂_S(L) for L={s↦||g_φ(z)-ε||²} using Maurer's vector-contraction inequality (Corollary 4 in [11]) with h_i(y)=||y-ε_i||². Compute the Lipschitz constant of h_i on {y: ||y||≤A} and verify whether the result is at least 2(A+√d) times the RHS of Eq. (15). Separately re-evaluate Bℓ: with ||g||≤A and ||ε||≤√d, the squared loss is bounded by (A+√d)², not A+√d. If either check confirms the missing factor, Theorem 3's stated bound is not derived as written; correcting the factors yields a valid but weaker bound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing gap is in Section 4, not only in the bounded-noise caveat. With loss ℓ(s)=||g_φ(z)-ε||² (Eq. 2), the proof sets Bℓ=A+√d immediately before Eq. (15), but this is a bound on ||g-ε||, not on its square. Under the stated assumptions, the correct almost-sure loss bound is Bℓ=(A+√d)². Similarly, the proof says the map y↦||y-ε_i|| is 1-Lipschitz and applies vector contraction; however, the function appearing in the loss class is y↦||y-ε_i||², which is 2(A+√d)-Lipschitz on the ball {||y||≤A}. Therefore the empirical Rademacher bound in Eq. (15), and consequently the key bound in Eq. (19), is missing a multiplicative factor proportional to A+√d. The theorem as stated does not follow from the preceding derivation. The unbounded-Gaussian-noise mismatch is real and is explicitly deferred to future work in the conclusion, but the norm/square conflation is an internal correctness issue that is at least as load-bearing for the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a data augmentation strategy for training generative models to produce initial guesses for parametric optimization: instead of keeping only converged solutions from projected gradient descent (PGD) runs, it retains the last k+1 iterates ('k-neighborhood paths'). For one-sided box-constrained quadratic programs, the paper proves a contraction property for PGD and derives a Rademacher-complexity generalization bound for a two-layer neural-network noise predictor in a DDPM-style loss. The bound is meant to quantify how the number of solver runs N, the contraction factor ρ, the neighborhood radius r_k, and the discount γ affect generalization. The paper also illustrates PGD trajectories for interior and active-constraint optima and discusses connections to the authors' GLENS method.","tokens_in":7024,"tokens_out":9751,"duration_ms":111591,"significance":"If the bound were correct as stated, it would provide a useful theoretical account of why solver-iterate data can help generative modeling for optimization: the bound explicitly shows how early-iterate radius, contraction, and discounting enter the sample complexity. The contraction proof (Theorem 1) is elementary and correct, and the overall idea of decomposing the path loss by iterate index is conceptually valuable. The paper is also transparent about several limitations, including the bounded-noise assumption and the need for tighter NN bounds. However, the central generalization bound contains a load-bearing norm-versus-squared-loss error, so the main theorem as written is not established.","major_comments":[{"comment":"The loss in Eq. (2) is ||g_phi(z)-epsilon||^2. The text states B_l = A + sqrt(d) after bounding ||g_phi(z)|| <= A and ||epsilon|| <= sqrt(d). This is a bound on the norm, not on the squared norm; the correct almost-sure bound is B_l = (A+sqrt(d))^2. The same issue appears in the vector-contraction step: the map y -> ||y-epsilon_i|| is 1-Lipschitz, but the function entering the loss class is y -> ||y-epsilon_i||^2, which on { ||y|| <= A } is 2(A+sqrt(d))-Lipschitz. Consequently Eq. (15) and its path version Eq. (19) are missing a multiplicative factor proportional to A+sqrt(d), and the confidence term in Eq. (18) should use the squared bound. As written, Theorem 3 does not follow from the proof. This is fixable by carrying the Lipschitz constant and squared bound through the derivation, but it is load-bearing for the paper's central claim.","section":"Section 4, Eq. (15) and Theorem 3 (Eqs. (18)-(19))"},{"comment":"The theorem assumes ||epsilon||_2 <= sqrt(d) almost surely. This is incompatible with the DDPM loss in Eq. (2), where epsilon ~ N(0, I_d) is Gaussian and therefore unbounded. The stated bound thus does not apply to the actual training distribution of the diffusion model described in Section 2. The paper acknowledges this only in the final sentence of the conclusion ('extend the analysis to unbounded Gaussian diffusion noise'). This restriction should be stated prominently when the bound is advertised as applying to diffusion models, or the theorem should be extended to unbounded noise. As it stands, the practical scope of the central result is narrower than the presentation suggests.","section":"Section 4, first paragraph after Eq. (14)"}],"minor_comments":[{"comment":"The derivation of Eq. (15) is very compressed: it invokes 'scalar contraction, the Efron-Stein inequality, and Jensen's inequality' without giving the intermediate steps or the source of the constant sqrt(12). Adding a short appendix or a fuller derivation would help verify the constants.","section":"Section 4, Eq. (15)"},{"comment":"The term 'k-neighborhood' may be misleading: membership in N^k_Ttheta is defined by the residual condition ||T^{k-1}(x)-T^k(x)|| <= tol, not by distance to the optimum. The later use of r_k(theta) as a radius clarifies the intent, but the initial definition could be phrased more directly as the set of points that meet the convergence criterion after k steps.","section":"Definition 1"},{"comment":"The acronyms DDDAS and 'InfoSymbiotic Systems' appear without definition in the abstract; a reader outside the DDDAS community will not know what they refer to. Define them at first use or relegate them to the body.","section":"Introduction and Abstract"},{"comment":"Reference [10] (GLENS) is described as 'our work' and as 'a new data-efficient global search method.' Since GLENS is not described or evaluated in this manuscript, the connection is programmatic rather than substantive. Consider framing this as future/related work rather than as a contribution of the present paper.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The norm/square error in Theorem 3 is real and directly undermines the stated bound, but it appears fixable within the manuscript's scope. The bounded-noise mismatch is also real, though the authors are transparent about it. If the authors correct the Lipschitz factor and squared bound, and restate the bounded-noise limitation more prominently, the paper could become acceptable. The theoretical contribution is otherwise sound in its broad approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper gives the first formal generalization bound for a genuinely practical trick—training generative models on the last k+1 PGD iterates from each solver run instead of just the converged solution—and the bound, if correct, cleanly explains how the contraction factor and the neighborhood radius trade off. But the central theorem as written isn't derived: the proof conflates the squared DDPM loss with its unsquared norm, which leaves Eq. (15) and (19) missing a factor of order A+√d. That's load-bearing but repairable.\n\nWhat's new: the k-neighborhood definition and the Rademacher bound with discount, contraction, and radius terms. This is a legitimate extension of standard generalization theory, and it reduces to the usual bound when k=0. The contraction theorem itself is textbook, but using it inside the Rademacher analysis is a fresh application. The vector-contraction inequality is applied correctly apart from the Lipschitz constant issue.\n\nThe main issue is in Section 4. The loss is ℓ(s)=||g_φ(z)−ε||². The proof sets B_l = A+√d, which bounds the norm, not the square. The correct bound is (A+√d)², and the map y↦||y−ε_i||² is 2(A+√d)-Lipschitz on the ball, not 1-Lipschitz. So the empirical Rademacher bound in Eq. (15), and hence the key bound in Eq. (19), is missing that factor. This is an internal correctness problem, at least as important as the bounded-noise caveat.\n\nThe bounded-noise assumption is genuinely limiting: DDPM uses unbounded Gaussian noise, and the paper explicitly defers that to future work. That's honest but means the main theorem doesn't cover the actual training distribution. There are no experiments beyond two illustrative PGD trajectories, which is fine for a theory paper but limits significance.\n\nCredit: the paper is self-contained, doesn't fit parameters to data, and the contraction proof is correct. The self-citation to GLENS is for motivation only and doesn't make the bound circular. The k-neighborhood bound itself is not in the cited literature.\n\nBottom line: a solid theoretical core with a fixable constant error. I'd send it to peer review—a serious referee would catch the squared-loss issue quickly—but the authors need to correct B_l and the Lipschitz factor before it's citable.","headline":"Nice new bound for a useful data-augmentation idea, but Theorem 3's proof has a squared-loss misstep that needs a constant fix.","tokens_in":7465,"tokens_out":3157,"would_cite":false,"duration_ms":32099,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68Q32","90C20","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"For box-constrained QPs solved by projected gradient descent, the paper proves a Rademacher bound showing how the solver's contraction rate, the discount factor, and the retained-neighborhood radius control the generalization gap when train","keywords":["k-neighborhood","Rademacher complexity","generalization bound","projected gradient descent","solver iterates","diffusion models","box-constrained quadratic programs","generative learning"],"falsifier":"Compute the generalization gap empirically for a box-constrained QP whose solver-iterate paths are trained with standard Gaussian diffusion noise; because P(||epsilon||_2 > sqrt(d)) > 0 for every finite d, the theorem's condition is violated, so a gap that reliably exceeds the Theorem 3 bound would confirm the bounded-noise premise is load-bearing. A cleaner test: truncate the noise to the sqrt(d) ball, train the two-layer class G, and check whether the measured worst-case gap respects the bound.","tokens_in":6601,"feed_emoji":"🧮","tokens_out":9951,"duration_ms":102467,"temperature":0.7,"pith_summary":"Generative models that propose initial guesses for parametric optimization problems are data-hungry: collecting enough converged solutions requires many expensive solver runs. This paper's central claim is that the intermediate iterates those runs already produce — normally thrown away — can be retained as extra training data, and that for one-sided box-constrained quadratic programs the benefit is analyzable. It proves that projected gradient descent contracts toward the optimum with factor rho = (lambda_max - lambda_min)/(lambda_max + lambda_min), then uses Rademacher complexity to bound the generalization gap of a two-layer noise-prediction network trained on 'k-neighborhood paths' (the final converged solution plus the preceding k iterates). The bound, Theorem 3, makes explicit how the number of solver runs N, the discount gamma, the contraction factor rho, and the neighborhood radius r_k enter: the gap decays as O(N^{-1/2}) with a coefficient that is the discounted average of radii R0 + rho^{k-j} r_k. If correct, this turns a data-augmentation trick into a quantitative design rule for data-collection strategies in learned optimization.","feed_headline":"Bound quantifies the value of reusing solver iterates","feed_subtitle":"For quadratic programs solved by projected gradient descent, the bound shows how many intermediate solver steps to keep.","key_machinery":"The load-bearing objects are the k-neighborhood (the set of initial points whose iterates meet the convergence tolerance within k steps), its radius r_k(theta), and the contraction factor rho = (lambda_max - lambda_min)/(lambda_max + lambda_min) of projected gradient descent. Training is organized into k-neighborhood paths: ordered tuples of the final converged solution and the preceding k iterates for each solver run. The analysis uses the discounted path loss J_i(phi) = (1/(k+1)) sum_{j=0}^k gamma^j loss(iterate_j), which lets the bound express the cost of earlier, more distant iterates through gamma^j and rho^{k-j} r_k. The generalization argument is carried by the empirical Rademacher co","core_discovery":"The core discovery is Theorem 3. For the family of one-sided box-constrained quadratic programs, projected gradient descent satisfies the contraction inequality ||T(x) - x*|| <= rho ||x - x*|| with rho = (lambda_max - lambda_min)/(lambda_max + lambda_min) < 1. Consequently, if the last k+1 iterates of a run are collected into a k-neighborhood path, the j-th-from-final iterate lies within distance rho^{k-j} r_k of the optimum. The paper then shows that, for a two-layer neural network class with bounded weights and activations, the empirical Rademacher complexity of the discounted path-loss class is at most sqrt(12) A L1 W sqrt(d) / sqrt(N) times the average over j of gamma^j (R0 + rho^{k-j} r","pith_inferences":["An immediate practical rule follows from the bound: stop retaining earlier iterates once rho^{k-j} r_k falls near the noise floor of the diffusion model, because further points only enlarge the average radius while the 1/sqrt(N) term stays fixed.","The proof only uses nonexpansiveness of projection and strong convexity, so the same machinery likely extends to other linearly convergent projection or proximal methods; if that extension holds, k-neighborhoods become a general template for augmenting solver data.","The i.i.d. path assumption is the main gap between theory and practice: real parametric families draw correlated instances, and accounting for that correlation could plausibly yield faster rates than N^{-1/2}, but the paper does not address it.","The bounded-noise assumption is violated by standard diffusion training; a truncation argument that conditions on the Gaussian noise landing in the sqrt(d) ball would turn the bound into a high-probability statement for the actual training distribution, at the cost of an extra confidence term."],"forward_implications":["If Theorem 3 holds, the train-to-test gap for learning PGD solver iterates decays as O(N^{-1/2}) in the number of independent solver runs, even though each path contains correlated points; k-neighborhoods increase the data per run but do not change this rate.","The bound isolates a concrete tradeoff: increasing k adds more terms to the discounted average, but the weight gamma^j and the contraction factor rho decide how much those earlier, more spread-out iterates contribute, so with rho near 1, early iterates are expensive to include.","Setting gamma < 1 yields a smaller Rademacher term than gamma = 1, so downweighting older iterates is provably beneficial within this class of discounted path losses.","The contraction result itself is a contribution: for one-sided box-constrained QPs, PGD's per-step error contraction is exactly rho, independent of whether the optimum lies in the interior or on an active upper bound.","Because the neighborhood radius r_k is defined through the convergence tolerance, tightening the tolerance shrinks r_k and therefore tightens the generalization bound, giving a principled trade between solver accuracy and sample complexity."],"fun_headline_variants":["Theorem quantifies data gain from PGD iterates","Keep solver iterates: bound proves it helps","Contraction bound: why reusing PGD steps works","Intermediate iterates: theory says use them","PGD iterate reuse justified by new bound"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The theorem assumes the diffusion noise vectors satisfy ||epsilon||_2 <= sqrt(d) almost surely; the denoising diffusion probabilistic model uses Gaussian noise with unbounded norm, so the loss class is unbounded and the stated bound does not apply to the actual training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Theorem quantifies data gain from PGD iterates","Keep solver iterates: bound proves it helps","Contraction bound: why reusing PGD steps works","Intermediate iterates: theory says use them","PGD iterate reuse justified by new bound"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1223,"prompt_tokens":722,"completion_tokens":501,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":466,"completion_tokens_details":{"reasoning_tokens":429}},"tokens_in":466,"tokens_out":501,"duration_ms":6286,"temperature":1.0,"reasoning_tokens":429,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T04:37:53.386132+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the generalization gap empirically for a box-constrained QP whose solver-iterate paths are trained with standard Gaussian diffusion noise; because P(||epsilon||_2 > sqrt(d)) > 0 for every finite d, the theorem's condition is violated, so a gap that reliably exceeds the Theorem 3 bound would confirm the bounded-noise premise is load-bearing. A cleaner test: truncate the noise to the sqrt(d) ball, train the two-layer class G, and check whether the measured worst-case gap respects the bound.","supporting_citations":[],"review_version":1}