{"id":"c21477ca-45dd-43ab-9d83-b8218982f9e8","arxiv_id":"2501.16573","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A proxy neural network, regularized by noise and a loss penalty, approximates the inverse-problem loss landscape and, in a two-step BFGS scheme, recovers unknown parameters more accurately than direct optimization on the true landscape.","lead":"The authors train neural networks to imitate the jagged loss landscapes used to compare physics simulations with data, then run BFGS on the smoothed imitation to find unknown parameters. The approach reports better parameter recovery than direct BFGS on three benchmark inverse problems, including a chaotic PDE.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's success depends on unverified basin preservation by regularization, with hyperparameters seemingly selected on the test set; Eq. (4) does not match the described penalty.","rationale":"The reader's weakest assumption and my concern overlap: the regularization must preserve the basin of the true optimum for the two-step scheme to work. However, I emphasize two additional issues that make the assumption currently untestable. First, Eq. (4) contradicts the described penalty condition; a constant multiplier mu does not create a low-region bias, so either the equation or the text is wrong, and the experiments rest on an unspecified loss. Second, the paper selects the 'top-performing' regularized network on the same problems used for evaluation, with no reported validation split. Since Figure 4 shows strong dependence on sigma and a failure regime at high sigma, tuning on the test set is not a minor stylistic issue; it is the difference between demonstrating a general method and demonstrating that favorable hyperparameters can be picked after seeing the answers. My proposed check, validation-based selection plus matched BFGS restarts, would settle this. I do not think the paper should be rejected: the proxy-loss idea is plausible and the figures provide useful evidence, but code, a principled hyperparameter selection protocol, and a compute-matched baseline are needed to establish the claim. The conditional verdict remains appropriate.","tokens_in":13427,"tokens_out":5344,"duration_ms":59528,"concrete_test":"For each system, split the inverse problems into training, validation, and test sets before any hyperparameter tuning; select (sigma, mu) to maximize validation accuracy only, then freeze and report test accuracy curves and resimulation error. Also report a BFGS baseline with the same number of restarts or true-loss evaluations as the two-step proxy method. If the ProxyNN+BFGS improvement disappears under this protocol, the central claim is an artifact of test-set selection or an unfair compute comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim relies on an unverified assumption: BFGS on the regularized proxy's loss lands in the basin of the true global minimum, so that the secondary BFGS on the true loss converges. Nothing in Sections 2.3-2.4 establishes this. The loss-penalty term is not specified correctly: the prose conditions the penalty on f_theta > L, but Eq. (4) simply multiplies the entire regression loss by mu, which for fixed mu is a constant scaling and cannot create the claimed low-region bias. Thus the actual regularization used in the experiments is underdetermined. Moreover, Section 3.3 reports results for the 'top-performing regularized proxy network,' and Table 1 lists sigma, mu per system, but no validation split is described. Choosing regularization strength after seeing test accuracy means the reported improvement, e.g., convergence 'almost doubles' in 2D billiards, may reflect test-set selection rather than a general property of smoothed proxy landscapes. Figure 4 itself shows that at sigma = 0.05 the basin disappears and gains vanish, so the regime where the claim holds is not characterized without a principled selection rule.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ProxyNN, a neural network that learns the configuration loss landscape of physics-based inverse problems, with implicit and explicit regularization (input noise and a loss penalty) intended to smooth the landscape. A two-step optimization scheme is then used: BFGS is applied first to the regularized ProxyNN-predicted loss, and the resulting point is refined by a second BFGS run on the ground-truth configuration loss. The method is evaluated on three inverse problems: the inviscid Burgers equation, the Kuramoto-Sivashinsky equation, and 2D and 4D billiards setups. The authors report that ProxyNN-based optimization improves convergence accuracy over BFGS and gradient descent on the ground-truth loss, with the strongest claim being that convergence 'almost doubles' in the 2D billiards setup.","tokens_in":13671,"tokens_out":5402,"duration_ms":49759,"significance":"If the results hold, the paper would offer a conceptually appealing way to leverage neural network regularization to ease optimization of chaotic, multi-modal inverse loss landscapes, and it identifies a concrete two-step optimization protocol that could be useful in PDE-constrained inverse problems. The manuscript formulates a clear generalized configuration-loss framework, evaluates on several nontrivial benchmarks, and transparently acknowledges in Section 6 that the relationship between regularization hyperparameters and landscape complexity is empirical. The central idea is plausible and the experimental evidence is suggestive, but the evaluation protocol as currently written leaves the strength of the claim uncertain and would need to be tightened for the paper's conclusions to be fully supported.","major_comments":[{"comment":"The regularization described in the prose does not match Eq. (4). The text states that the training loss is scaled with µ > 1 for all samples where fθ(Y*, Xs) > L, which is supposed to create a geometric bias favoring low-lying regions of the loss landscape. Eq. (4), however, multiplies the entire squared error by µ, which for a fixed µ is a constant rescaling of the loss and cannot create the described bias toward regions where L is small. If the implementation used a conditional penalty, the equation must be corrected; if it used Eq. (4) as written, the claimed mechanism for loss-penalty regularization is not present in the paper. The actual regularization used in the experiments is therefore underdetermined.","section":"§2.3, Eq. (4)"},{"comment":"The reported convergence improvements are based on the 'top-performing regularized proxy network' for each system, with the corresponding {σ, µ} listed in Table 1. No validation split or selection rule is described, so the hyperparameters appear to have been chosen by looking at the test-set accuracy curves in Figure 5. This makes the headline improvement (e.g., 'almost doubles' in 2D billiards) the result of post-hoc selection on the benchmark set rather than evidence for a general property of smoothed proxy landscapes. A principled selection criterion (e.g., choose σ and µ on a separate validation subset, or by a criterion that does not use the known X*) and then reporting accuracy on held-out problems is needed to support the central claim.","section":"§3.3, §4, Table 1"},{"comment":"All optimization accuracy numbers are point estimates. There are no error bars or multiple seed runs for the ProxyNN training or for the BFGS optimization, even though the training involves stochasticity (Adam, random sampling of Xs) and the results are likely sensitive to initial guesses. Without a measure of variance, it is unclear whether the reported advantage of ProxyNN over BFGS, which is described as 'almost doubles,' is statistically significant. The manuscript should report mean and standard deviation across at least several independent training seeds and initial-guess choices.","section":"§3.3, §4, Figures 4-6"},{"comment":"The initial-guess protocol for BFGS is not specified. For the 256 unique inverse problems in each setup, it is not stated how the starting point is generated for (i) the baseline BFGS on the ground-truth loss, (ii) the primary BFGS step on the ProxyNN loss, and (iii) the secondary step. If the initial guess for the baseline is not the same as that used in the ProxyNN pipeline, the comparison may be unfair. The manuscript must state the initialization distribution and confirm that the same protocol is used for all methods.","section":"§2.4, §3.3"}],"minor_comments":[{"comment":"The text contains 'Burgerséquation' with a stray accent; it should read 'Burgers equation'.","section":"§3.1"},{"comment":"The sentence 'A summarize the network configurations...' is ungrammatical; it should read 'Appendix A summarizes the network configurations...'.","section":"§3.2"},{"comment":"The second argument of L is written as Ys, while Eq. (1) defines L as a function of Xs; please make the notation consistent throughout.","section":"Eq. (4)"},{"comment":"BFGS is described as a 'momentum-based optimizer.' BFGS is a quasi-Newton method, not a momentum-based method, and this terminology should be corrected.","section":"Abstract and §1"},{"comment":"The claim that convergence 'almost doubles' should be quantified with a specific threshold and the underlying accuracy values, e.g., 'at threshold e=0.05, accuracy increases from A% to B%.'","section":"§4"},{"comment":"The subcaptions in Figure 2 are garbled (e.g., '(y0), '(y0) /30'); they should be cleaned up for readability.","section":"Figure 2"},{"comment":"The 'Sampling Rate' column is used without a definition; it should be explained how often control parameters are sampled per trajectory in each setup.","section":"Appendix A, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The core idea of learning and smoothing the configuration loss landscape is within the journal's scope and potentially interesting. The main risks are the evaluation circularity (test-set hyperparameter selection), the discrepancy between the described regularization mechanism and Eq. (4), the lack of variance estimates, and the unspecified initialization protocol for BFGS. All of these are fixable within the scope of a major revision, and the paper should be reconsidered after those points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe core idea here is plausible, but the evidence is weaker than the abstract suggests. The paper trains a deep network (ProxyNN) to predict the configuration loss of an inverse problem, adds noise and loss-penalty regularization to smooth the predicted landscape, then runs BFGS first on the proxy and then on the true loss. The surprising claim is that this two-step scheme roughly doubles convergence accuracy on a 2D billiards problem compared to BFGS on the true loss.\n\nWhat's genuinely new is the specific combination of training a proxy to mimic the loss while deliberately simplifying the landscape, then using it as a warm start. The idea of smoothing an objective to avoid local minima is not new, and the work is an incremental extension of neural adjoint methods, but the demonstration on Burgers, KS, and billiards is a useful empirical addition. The paper also includes a candid limitations section, which is a plus.\n\nThe soft spots are real. First, the loss-penalty regularization as described in the text says it penalizes only where the proxy overestimates the true loss (f_theta > L), but Eq. (4) just multiplies the entire regression loss by a constant mu, which cannot create that conditional bias. Either the equation is wrong or the actual implementation is different; the paper doesn't say which. This underdetermines the method.\n\nSecond, Section 3.3 reports results for the \"top-performing regularized proxy network,\" and Table 1 lists sigma and mu per system, but nothing describes a validation split. Choosing regularization strengths after seeing test accuracy means the headline improvements, e.g., \"almost doubles\" in 2D billiards, could reflect test-set selection rather than a robust property. Figure 4 itself shows that at sigma=0.05 the basin disappears and gains vanish, so the working regime is narrow.\n\nThird, there are no error bars across seeds and no explicit initial-guess protocol for BFGS, both of which matter for convergence claims. The paper's reliance on the proxy's smoothed landscape preserving the global basin is an empirical assumption that is tested only on the three chosen problems.\n\nThese issues are significant but not fatal to the idea. The core notion deserves scrutiny. I would send this to a serious referee, asking for a precise statement of the loss-penalty, a validation-based hyperparameter selection, error bars, and code. For a reader working on surrogate optimization, the paper is worth a coffee discussion but not yet a solid citation.\n\nMy recommendation: accept for peer review with major revision.","headline":"A plausible surrogate-optimization idea undercut by suspicious hyperparameter selection and an ambiguous loss-penalty; deserves referee time but not yet a citation.","tokens_in":14161,"tokens_out":2888,"would_cite":false,"duration_ms":26274,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimizing on a neural-network-smoothed copy of a chaotic loss landscape gets BFGS to the true inverse parameters more often than optimizing on the raw landscape.","keywords":["inverse problems","configuration loss landscape","proxy neural network","regularization","BFGS","partial differential equations","chaotic systems","billiards"],"falsifier":"Choose a single inverse problem and vary the noise scale $\\sigma$ until the proxy-predicted global minimum moves more than one BFGS tolerance away from the true parameters; the two-step method will then return a wrong answer even with unlimited secondary iterations. A direct check is to plot the proxy landscape's argmin against $\\sigma$ and look for a systematic drift.","tokens_in":13241,"feed_emoji":"🎯","tokens_out":6519,"duration_ms":54153,"temperature":0.7,"pith_summary":"The paper argues that the failure of gradient-based optimizers on physics inverse problems stems from the chaotic, multi-minima shape of the configuration loss landscape rather than from the inverse problem itself. It trains a proxy neural network to predict this loss from a target trajectory and a candidate parameter set, then regularizes the network during training to flatten the landscape while keeping the global minimum in place. A two-step scheme—first BFGS on the smoothed proxy loss, then BFGS on the true loss—converges to the true parameters more often than BFGS applied directly to the true loss on Burgers, Kuramoto-Sivashinsky, and billiards problems. The reader should care because this turns a learned surrogate into a tool for optimization, not just prediction.","feed_headline":"Proxy-smoothed losses speed convergence for physics inverse problems","feed_subtitle":"Training a network to imitate the chaotic loss landscape lets BFGS skip local minima and hit the true parameters.","key_machinery":"The configuration loss $L(Y^*, X_s) = \\|\\mathcal{P}(Y_0, X_s) - Y^*\\|_2^2$ measures how far a trajectory evolved under candidate parameters $X_s$ is from the observed target trajectory $Y^*$. The proxy network $f_\\theta(Y^*, X_s)$ is trained to predict $L$ using Adam on a regularized objective $L^R_N = \\mu \\|f_\\theta(Y^*, X_s + \\sigma N) - L(Y^*, Y_s)\\|^2$, where $\\sigma N$ adds Gaussian input noise and $\\mu > 1$ penalizes predictions above the true loss. These two pressures smooth high-frequency features of the learned landscape while favoring low-lying regions near minima. The optimization phase runs BFGS first on the proxy-predicted landscape, then on the true configuration loss starting from the proxy's result.","core_discovery":"Proxy neural networks can replicate the configuration loss $L(Y^*, X_s) = \\|\\mathcal{P}(Y_0, X_s) - Y^*\\|_2^2$ across spatio-temporal trajectories, and with noise and loss-penalty regularization they produce smoother versions of the landscape whose global minimum still tracks the true parameters. Optimizing this smoothed proxy with BFGS lands near the true optimum, and a second BFGS step on the ground-truth loss refines the result. In the 2D billiards setup, convergence accuracy almost doubles compared with BFGS on the ground-truth loss.","pith_inferences":["A natural extension the paper does not pursue is to apply the same two-step idea to other optimizers (e.g., Adam or Newton methods) by training the proxy to output a smoothed loss for any candidate parameters, effectively adding a tunable smoothing knob to the optimizer.","The asymmetric loss penalty ($\\mu > 1$ when the proxy over-predicts) biases the learned landscape toward its low regions; a symmetric penalty or a gradient-norm penalty might yield the same smoothing with different trade-offs between smoothness and basin preservation, which the paper does not test.","Because the proxy is trained in tandem with the numerical solver, the method inherits the solver's accuracy; coupling the primary step to a differentiable solver would allow end-to-end gradient flow from the coarse-smoothed landscape to the fine true loss."],"forward_implications":["The two-step proxy optimization improves convergence accuracy over BFGS and gradient descent on the ground-truth loss across all three tested systems (Burgers, Kuramoto-Sivashinsky, 2D and 4D billiards).","In the 2D billiards setup, convergence to the optimal solution almost doubles when the primary step uses the proxy-predicted loss rather than the ground-truth loss.","Resimulation error, the L2 distance between trajectories from predicted and true parameters, is lower for proxy-optimized parameters than for baseline optimizers.","The regularization strength controls a trade-off: too little leaves the chaotic landscape intact, too much produces an oversimplified landscape with poor convergence.","ProxyNNs trained with Fourier feature inputs generalize well enough to predict loss values for unseen target trajectories sampled from the same initial state."],"supporting_citations":[{"why":"supplies the implicit/explicit regularization perspective used to control network complexity during ProxyNN training","marker":"Dherin et al. (2022)"},{"why":"provides the Gramacy & Lee test function used to illustrate how local minima cause piecewise BFGS convergence","marker":"Gramacy & Lee (2010)"},{"why":"Fourier feature mapping used as network input to reduce spectral bias and improve learning of high-frequency loss landscapes","marker":"Tancik et al. (2020)"},{"why":"differentiable billiards simulation that the paper adapts to build the 2D and 4D inverse problems","marker":"Hu et al. (2020)"},{"why":"defines the BFGS quasi-Newton method that forms both the baseline and the optimizer used in the two-step scheme","marker":"Broyden et al. (1970)"},{"why":"Adam optimizer used for training the ProxyNN","marker":"Kingma & Ba (2017)"}],"fun_headline_variants":["Proxy-smoothed loss landscapes double convergence accuracy for physics inversions","Neural proxy smoothing helps optimizers skip local minima in physics inverse problems","Training a proxy network to mimic loss landscapes improves BFGS convergence","Smoothing chaotic loss landscapes with neural proxies nearly doubles inversion accuracy","Proxy networks flatten chaotic physics loss landscapes, improving inverse problem accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The regularized proxy must keep the global minimum at the true parameters while smoothing the landscape; if the smoothing shifts or erases that basin, the primary BFGS step lands in the wrong region and the secondary step cannot recover.","fun_headline_variants_meta":{"raw":{"variants":["Proxy-smoothed loss landscapes double convergence accuracy for physics inversions","Neural proxy smoothing helps optimizers skip local minima in physics inverse problems","Training a proxy network to mimic loss landscapes improves BFGS convergence","Smoothing chaotic loss landscapes with neural proxies nearly doubles inversion accuracy","Proxy networks flatten chaotic physics loss landscapes, improving inverse problem accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000928,"raw_usage":{"total_tokens":3895,"prompt_tokens":788,"completion_tokens":3107,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":404,"completion_tokens_details":{"reasoning_tokens":3017}},"tokens_in":404,"tokens_out":3107,"duration_ms":21625,"temperature":1.0,"reasoning_tokens":3017,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T12:11:38.901038+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Choose a single inverse problem and vary the noise scale $\\sigma$ until the proxy-predicted global minimum moves more than one BFGS tolerance away from the true parameters; the two-step method will then return a wrong answer even with unlimited secondary iterations. A direct check is to plot the proxy landscape's argmin against $\\sigma$ and look for a systematic drift.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the implicit/explicit regularization perspective used to control network complexity during ProxyNN training"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the Gramacy & Lee test function used to illustrate how local minima cause piecewise BFGS convergence"},{"cited_title":"Convergence properties of a class of quasi-newton methods in optimization","cited_arxiv_id":null,"evidence_quote":"defines the BFGS quasi-Newton method that forms both the baseline and the optimizer used in the two-step scheme"}],"review_version":1}