{"id":"97dba58c-78f6-4cc2-9c49-b8112cc58027","arxiv_id":"2505.11342","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding solver sensitivity information to the training loss of optimization proxies reduces prediction error and constraint violations on AC-OPF benchmarks and improves self-supervised portfolio proxies in the medium-risk regime, though the theory and experiments only partially align.","lead":"This paper trains neural-network proxies for optimization problems with an extra loss term that matches the solver's derivatives, not just its solutions. The method reduces prediction error and constraint violations on power-grid benchmarks and improves a self-supervised portfolio task, but its theoretical guarantee assumes more than the experiments deliver.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3's exact-interpolation, full-Jacobian, C^2 assumptions do not hold for the masked, finite-sample ReLU/LeakyReLU models actually trained and tested.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: Theorem 3 requires exact value and full-Jacobian interpolation by a C^2 proxy, while the implemented method uses randomly masked Jacobians, finite-sample losses, and ReLU/LeakyReLU/Sigmoid activations. My reading confirms this and adds that the AC-OPF solution mapping itself is only piecewise smooth, so even the regularity side of Assumption 2 is questionable for the main benchmarks. Section 7 explicitly admits that irregular cases occurred and that masking was introduced because dense Jacobian supervision degraded performance, which strengthens the claim that the theoretical regime and the practical regime are distinct. The theorem's proof is internally sound, and the empirical results may well be genuine, but the paper does not provide evidence that the uniform O(delta^2) guarantee applies to the models whose gains are reported. A conditional verdict remains appropriate: the authors should either adapt the theory to masked, finite-sample, nonsmooth settings or provide experiments that operate in the theorem's regime. No ad hominem is intended; the issue is a mismatch between assumptions and implementation, not a question of integrity.","tokens_in":12778,"tokens_out":4449,"duration_ms":51090,"concrete_test":"Retrain the IEEE-300 (or PEGASE-1k) proxy with 100% dense Jacobian supervision and tanh/softplus activations, using an overparameterized network and a training schedule that drives the maximum value and Jacobian residuals on the training set to numerical zero; then measure test MSE and infeasibility, and check whether test error scales roughly as delta^2 when the training-set covering radius is varied. If the dense, C^2, near-exact-interpolation model fails to reproduce the reported improvements, or if its error does not follow the predicted O(delta^2) trend, then the empirical gains are not attributable to the mechanism Theorem 3 describes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central theoretical result, Theorem 3, assumes that 'Sobolev training enforces \\hat g(pi)=g(pi), D\\hat g(pi)=Dg(pi), for all pi in T' and, per Remark 1, that the proxy has C^2 activations with bounded curvature. The method actually implemented (Sections 3 and 5, Table 3) minimizes a finite-sample, weighted combination of value loss and randomly masked Jacobian loss keeping only 5-25% of Jacobian entries, using ReLU, LeakyReLU, or Sigmoid activations. Therefore neither exact value-and-Jacobian interpolation, nor full Jacobian matching, nor C^2 smoothness holds for any model whose results are reported. Moreover, AC-OPF solution maps are only piecewise smooth because active sets change with parameters, and Section 7 concedes that 'irregular cases were present.' Consequently, the O(delta^2) bound does not govern the experiments; the reported gains could stem from the masking/regularization scheme or from the value-loss term rather than from the derivative-interpolation mechanism the theorem analyzes. The paper's own Section 7 acknowledges that dense Jacobian supervision overconstrained the model and that random masking was introduced to fix it, confirming that the exact-interpolation regime of Theorem 3 is not the regime tested. The proof itself appears correct under its stated assumptions, but those assumptions are disconnected from the method as implemented, so the headline theoretical guarantee is not established for the proposed algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a masked Sobolev training objective for optimization proxies, in which a standard value-regression loss is augmented by a sparsified Jacobian-matching term that aligns the proxy's derivatives with solver sensitivities. The authors claim uniform approximation guarantees, including an O(delta^2) bound under joint value and Jacobian interpolation, and report empirical results on three PGLib AC-OPF benchmarks plus a mean-variance portfolio task. In the supervised AC-OPF experiments, the Sobolev-trained proxy generally improves MSE and constraint-violation metrics over an MSE-only baseline; in the semi-self-supervised portfolio task, it improves average optimality gap in the medium-risk regime but not in the tight-risk regime.","tokens_in":13049,"tokens_out":7057,"duration_ms":71234,"significance":"If Theorem 3 applied to the actual trained models, the paper would make a substantial contribution by connecting derivative matching to a uniform approximation guarantee with a quadratic dependence on training-set density. The proof of Theorem 3 is short and mathematically correct under its stated assumptions, and the reported empirical gains on large PGLib instances are suggestive. However, the central theoretical guarantee is not connected to the method as implemented: the paper's own loss is finite-sample and mask-sparsified, the activations used in most experiments are not C^2, and the AC-OPF solution map is not globally C^1. These gaps mean the headline theoretical result currently serves mainly as motivation rather than as a guarantee for the evaluated algorithm.","major_comments":[{"comment":"The theorem assumes that Sobolev training enforces exact interpolation of both g(p_i) and Dg(p_i) at every training point, as stated in the paragraph before Assumption 1. The loss defined in Section 3 is a finite-sample weighted sum of a value term and a randomly masked Jacobian term, and no mechanism in the training procedure enforces exact interpolation. With the reported masks keeping only 5-25% of Jacobian entries, D\\hat{g}_theta(p_i) is not equal to Dg(p_i) for most entries, so Theorem 3 cannot be invoked for the models whose results are reported. Please either train with an interpolation-enforcing mechanism, extend the bound to the masked finite-sample objective with an explicit residual term depending on the training loss, or state clearly that Theorem 3 is an idealized result that does not govern the experiments.","section":"Section 4, Theorem 3"},{"comment":"Remark 1 restricts Theorems 1-3 to proxies with C^2 activations of bounded curvature, giving tanh and softplus as examples. Table 3 lists ReLU activations for PEGASE-1k and RTE-6k and LeakyReLU for the Markowitz model; these are not C^2, so the theoretical guarantee does not cover them. The sigmoid model on IEEE-300 is C^2, but it still does not satisfy the full-Jacobian interpolation assumption because of the masked Jacobian loss. Consequently, none of the four experimental configurations satisfies the assumptions needed for Theorem 3 as stated.","section":"Remark 1 and Table 3"},{"comment":"Theorem 3 requires g to be C^1 on the entire compact parameter set P, with a Lipschitz Jacobian. AC-OPF solution maps are only piecewise smooth because the set of active constraints can change as demand parameters vary, and Section 7 concedes that 'irregular cases were present' in the experiments. The theorem's conclusion is a global supremum bound, so it cannot tolerate nonsmooth regions even if such regions are rare. The statement that irregular cases 'did not materially affect overall proxy performance' is an empirical claim and does not repair the theoretical gap; the theory should either restrict P to a smooth region or be reformulated for piecewise-smooth maps.","section":"Assumption 2 and Section 7"},{"comment":"The portfolio study is described as self-supervised and the introduction to Section 6 states that SSL 'avoids the need for optimal solutions.' However, the Sobolev variant is trained on 'solver sensitivities,' and Equation (1) shows that these sensitivities are obtained by differentiating the KKT system at an optimal primal-dual solution. Computing Dg(p_i) therefore requires computing x*(p_i) and lambda*(p_i), so the method does not in fact avoid the need for optimal solutions. The paper should clarify what 'semi' self-supervised means, or explain how sensitivities can be produced without solving the optimization problem.","section":"Section 6"}],"minor_comments":[{"comment":"The 'Mask sparsity (%)' column lists the values 0.145, 0.305, 0.125, and 0.355, which are inconsistent with the usual meaning of a percentage and with the paper's statement that 5-25% of Jacobian entries are retained. Please clarify whether these numbers are retained fractions or sparsity percentages, and ensure they match the values used in Appendix D.","section":"Table 3"},{"comment":"The definition of RMI_i normalizes by max_{i,j}(Infeas^{Sobolev}_{i,j}), the largest violation of the Sobolev model only, rather than by a maximum over both models. This makes the reported relative improvements hard to interpret and highly sensitive to the scale of Sobolev violations; normalizing by the maximum over both baselines would be more natural.","section":"Section 5.2"},{"comment":"There are several typographical issues: 'intwo' in the abstract, 'refereed to' in Section 5.1, and 'Xases' in the caption of Figure 2(b). Please proofread the manuscript.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The main blocker is the gap between Theorem 3 and the implemented masked, finite-sample, ReLU/LeakyReLU models. This is fixable within the manuscript's scope by reframing Theorem 3 as an idealized statement with an explicit approximate-interpolation bound, or by changing the training procedure to enforce exact interpolation. I do not see the DiffOpt.jl/L2O.jl citations as circular, since the empirical comparison is against an MSE-only baseline and the theoretical argument does not depend on those packages. If the authors address the assumptions gap, the paper would be a reasonable contribution to the optimization-proxy literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The practical contribution is real: random masking of Jacobian entries makes Sobolev training scale to AC-OPF with thousands of buses, and the reported reductions in MSE and worst-case infeasibility are large enough to take seriously. The theory, however, is a clean interpolation bound that does not cover the method actually trained. Theorem 3 assumes exact value and full Jacobian interpolation on the training set and C^2 activations, while the experiments use ReLU/LeakyReLU and mask 75–95% of the Jacobian. So the headline guarantee is about a different algorithm than the one being tested.\n\nWhat is new: the sparse-masked Sobolev loss, the AC-OPF empirical study, and the self-supervised Markowitz study. The masking idea is smart and empirically motivated—the authors show dense Jacobian supervision hurts, and keeping only 5–25% of entries works best. That is a genuinely useful design insight for large-scale optimization proxies. The AC-OPF numbers (up to 56% MSE reduction, up to 4x reduction in median worst-case infeasibility) are striking, though the optimality gap increases slightly on two of three cases. The Markowitz result is honest: Sobolev loses in the tight-risk regime and wins outside it, and the authors explicitly propose a mixture-of-experts extension. Credit where due.\n\nSoft spots. The theory-practice gap is the main one. The proof of Theorem 3 is correct under its assumptions, but those assumptions are not met by the actual training procedure. The paper's own Section 7 concedes that irregular sensitivity cases were present and that dense Jacobian supervision overconstrained the model, which is exactly why masking was introduced. So Theorem 3 is a motivation, not a guarantee, for the reported results. Also missing: code, data, and error bars. The empirical claims are plausible but not independently checkable. Table 3's mask-sparsity and hyperparameter columns are garbled in the preprint, which doesn't help. The self-supervised study is a single task with one baseline; more ablations would be needed to judge generality.\n\nWho this is for: people working on learning-to-optimize, especially AC-OPF surrogates, and anyone building differentiable optimization layers. It deserves a serious referee because the empirical recipe and masking insight are useful, even if the theory needs reframing. Send it to review, but ask the authors to release code/data and either adapt the theorem to masked training or clearly mark the bound as idealized. I would not cite it in my own work until those pieces are in place.","headline":"A useful masking recipe for Sobolev-trained optimization proxies, with an elegant but disconnected theory; worth refereeing, but the guarantee does not cover the actual method.","tokens_in":13615,"tokens_out":3057,"would_cite":false,"duration_ms":31152,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","90C31","90C26"],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding solver-sensitivity matching to proxy training yields quadratic error bounds and cuts large-scale AC-OPF error by up to 56%.","keywords":["Sobolev training","optimization proxies","parametric optimization","sensitivity analysis","AC optimal power flow","mean-variance portfolio","Jacobian matching","self-supervised learning"],"falsifier":"On a problem with an analytically known solution map $g$ and known derivative-Lipschitz constant $M_g$, train a smooth network with exact value and dense Jacobian interpolation on training grids of two different covering radii $\\delta$; if the measured uniform error does not scale like $\\delta^2$, or exceeds $\\frac{1}{2}(M_g+M_{\\hat{g}})\\delta^2$, then Theorem 3 is not the mechanism behind the reported gains.","tokens_in":12537,"feed_emoji":"⚡","tokens_out":13751,"duration_ms":120913,"temperature":0.7,"pith_summary":"This paper tries to establish that an optimization proxy—a neural network trained to imitate the solution map of a parametric optimization problem—improves markedly when its loss also matches the solver's local derivatives, not just its outputs. The paper proves that if the proxy reproduces both the value and the derivative matrix of the true solution map at every training point, and both maps have Lipschitz derivatives, the worst-case approximation error is at most a constant times the square of the training-set covering radius, a quadratic improvement over value-only matching. On three large alternating-current optimal power flow (AC-OPF) benchmarks, the supervised derivative-matching proxy cuts mean squared error by up to 56% and reduces median worst-case constraint violation by up to a factor of four while keeping the optimality gap below 0.22%. On a self-supervised mean-variance portfolio task, the same derivative signal halves the average optimality gap in the medium-risk region and matches the baseline in the tight-risk region, which matters because a single forward pass could then replace an iterative solver in time-critical or safety-critical decisions.","feed_headline":"Matching solver derivatives makes power-flow proxies up to 4x safer","feed_subtitle":"Power-flow proxies run in one forward pass, and lower violations make them safer for fast grid dispatch.","key_machinery":"The key object is the masked Sobolev loss, $L(\\theta)=\\frac{1}{N}\\sum_i \\ell(\\hat{g}_\\theta(p_i),g(p_i)) + \\frac{\\lambda}{N}\\sum_i \\ell_d(M\\odot D\\hat{g}_\\theta(p_i), M\\odot Dg(p_i))$, which combines value regression with a penalty on selected derivative entries. The proof engine is a first-order Taylor expansion with integral remainder: exact value and Jacobian interpolation at a training point cancels the constant and linear terms in $\\hat{g}_\\theta-g$, leaving only second-order remainders bounded by $\\frac{1}{2}(M_g+M_{\\hat{g}})\\|h\\|^2$, which yields the covering-radius-squared guarantee.","core_discovery":"The central claim is that solver sensitivities are a cheap, high-value training signal for end-to-end optimization proxies. Theorem 3 shows that matching both the solution value and its first-order derivative at every training point yields $\\sup_{p\\in P}\\|\\hat{g}_\\theta(p)-g(p)\\| \\le \\frac{1}{2}(M_g+M_{\\hat{g}})\\delta^2$, provided the true solution map and the network are $C^1$ with Lipschitz Jacobians on the compact parameter domain. In practice the loss matches only a randomly masked subset of Jacobian entries, and the paper argues that extreme sparsity stabilizes training and improves test error. Empirically, supervised derivative-matching training on AC-OPF is reported to cut mean squared error by up to 56% and median worst-case infeasibility by up to 400%, while the self-supervised portfolio experiment reports the average optimality gap in the medium-risk region dropping from 18.9% to 8.7%.","pith_inferences":["A direct testable extension is to embed the derivative-matching proxy in an outer optimization loop and measure convergence speed; the paper argues for gradient fidelity but never measures its downstream effect.","The sharp split between tight-risk and medium-risk portfolio regimes suggests training a gated model that delegates to the value-only proxy when the risk budget is below 10% of capital and to the derivative-matching proxy otherwise; the paper names this mixture-of-experts idea but does not implement it.","The quadratic error bound implies derivative labels could substitute for denser parameter sampling; a grid-refinement experiment comparing value-only and derivative-matching training at matched sample counts would quantify that trade-off."],"forward_implications":["Supervised derivative matching changes the accuracy-feasibility trade-off for AC-OPF proxies: it lowers mean squared error and sharply reduces worst-case violations while keeping optimality gaps under 0.22%.","Self-supervised derivative matching offers a path to train proxies without optimal labels; in the medium-risk portfolio region it halves the average optimality gap.","Matching derivatives reduces the sample complexity of learning the solution map, since the theoretical error depends on the square of the training-set radius rather than the radius itself.","Randomly masking most Jacobian entries is not only a memory saver; the paper's ablations indicate that dense Jacobian supervision actively hurts performance."],"supporting_citations":[{"why":"introduces Sobolev training for neural networks, the loss paradigm this paper adapts to optimization proxies","marker":"[4]"},{"why":"establishes KKT-based sensitivity analysis, the foundation for treating solver derivatives as training targets","marker":"[8]"},{"why":"supplies the Lipschitz-continuity guarantee for optimal solution mappings that underpins the uniform error bounds","marker":"[33]"},{"why":"provides the three large AC-OPF benchmark networks used in the supervised experiments","marker":"[7]"},{"why":"gives the differentiable optimization layer used to extract exact solver sensitivities in the data pipeline","marker":"[28]"},{"why":"supplies the nonlinear programming solver used to compute ground-truth solutions for the supervised tasks","marker":"[30]"}],"fun_headline_variants":["Derivative-matching training cuts AC-OPF proxy errors by 56% and violations by 4x","Sobolev training reduces grid proxy violations 4x","Self-supervised Sobolev proxies halve portfolio optimality gap","Sobolev training: 56% less error, 4x fewer violations in AC-OPF","Matching solver derivatives yields 4x safer optimization proxies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All theoretical guarantees rest on the premise that at every training point the proxy exactly reproduces both the solver's output and its full derivative matrix, using smooth activations with bounded curvature and a parametric problem regular enough for the solution map to be smooth; the paper's own experiments instead keep only 5-25% of Jacobian entries, use ReLU or LeakyReLU activations, and note that regularity conditions occasionally fail in AC-OPF, so the stated bound does not directly govern the reported models.","fun_headline_variants_meta":{"raw":{"variants":["Derivative-matching training cuts AC-OPF proxy errors by 56% and violations by 4x","Sobolev training reduces grid proxy violations 4x","Self-supervised Sobolev proxies halve portfolio optimality gap","Sobolev training: 56% less error, 4x fewer violations in AC-OPF","Matching solver derivatives yields 4x safer optimization proxies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001116,"raw_usage":{"total_tokens":4686,"prompt_tokens":1027,"completion_tokens":3659,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":3553}},"tokens_in":643,"tokens_out":3659,"duration_ms":28376,"temperature":1.0,"reasoning_tokens":3553,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:54:16.115359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a problem with an analytically known solution map $g$ and known derivative-Lipschitz constant $M_g$, train a smooth network with exact value and dense Jacobian interpolation on training grids of two different covering radii $\\delta$; if the measured uniform error does not scale like $\\delta^2$, or exceeds $\\frac{1}{2}(M_g+M_{\\hat{g}})\\delta^2$, then Theorem 3 is not the mechanism behind the reported gains.","supporting_citations":[{"cited_title":"Sobolev training for neural networks.Advances in neural information processing systems, 30, 2017","cited_arxiv_id":null,"evidence_quote":"introduces Sobolev training for neural networks, the loss paradigm this paper adapts to optimization proxies"},{"cited_title":"Sensitivity analysis for nonlinear programming using penalty methods","cited_arxiv_id":null,"evidence_quote":"establishes KKT-based sensitivity analysis, the foundation for treating solver derivatives as training targets"},{"cited_title":"Sensitivity analysis in nonlinear programs and variational inequalities via continuous selections.SIAM Journal on Control and Optimization, 33(4):1040–1060, 1995","cited_arxiv_id":null,"evidence_quote":"supplies the Lipschitz-continuity guarantee for optimal solution mappings that underpins the uniform error bounds"},{"cited_title":"Flexible differen- tiable optimization via model transformations.INFORMS Journal on Computing, 36(2):456– 478, 2024","cited_arxiv_id":null,"evidence_quote":"gives the differentiable optimization layer used to extract exact solver sensitivities in the data pipeline"},{"cited_title":"Large-scale nonlinear programming using Ipopt: An integrating framework for enterprise-wide dynamic optimization.Computers & Chemical Engineering, 33(3):575–582, 2009","cited_arxiv_id":null,"evidence_quote":"supplies the nonlinear programming solver used to compute ground-truth solutions for the supervised tasks"}],"review_version":1}