{"id":"ddd849cd-a528-424d-8f94-0ae7c73b3186","arxiv_id":"2602.06842","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"For hybrid neural-classical PDE solvers, matching the training loss to the solver dynamics and using residual-minimizing Anderson acceleration prevents convergence stagnation at false fixed points.","lead":"This paper shows that hybrid deep-learning PDE solvers can stall even when the network is well trained, and that fixing the update rule—not just the architecture—can resolve the stall. It introduces physics-aware Anderson acceleration, which minimizes the physical residual instead of the network update, and reports faster, more reliable convergence.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The false-fixed-point diagnosis rests on a plateau, not a proven fixed point; a longer-horizon run is needed to rule out slow convergence.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the false-fixed-point diagnosis is inferred from a plateau without ruling out slow convergence. This is the most central issue because the paper's mechanistic story and the motivation for PA-AA both rely on the iteration being truly trapped. If the test shows continued residual decay, the 'false fixed point' claim weakens, and PA-AA's benefit becomes one of acceleration rather than escaping a trap. The reader's CONDITIONAL verdict already accounts for this uncertainty, so no verdict adjustment is needed; the proposed check would either confirm or refute the concern.","tokens_in":10531,"tokens_out":6733,"duration_ms":72184,"concrete_test":"Re-run the exact HINTS/Diffusion N=801 experiment of Figure 1 for at least 10^5 full cycles, or until the update norm reaches ~1e-12, recording ||r_k|| each cycle. Fit log||r_k|| versus k: if the slope is negative and bounded away from zero over the last decade, the plateau is slow convergence; if the slope becomes zero while ||r_k|| stays above 1, a false fixed point is confirmed. Repeat with a tighter termination tolerance (e.g., 1e-12) and report relative residual; a single run without error bars or a longer horizon is insufficient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanistic claim—that DL-HIMs stagnate at false fixed points where the update vanishes while the residual remains large—is supported in Section 5.2 mainly by Figure 1, which shows the update norm plateauing near 1e-4 while the residual stays near 1e1. But an update norm of 1e-4 is not zero, and no long-horizon run is reported. If the residual is still decreasing, even slowly, then the phenomenon is slow convergence rather than a false fixed point. This matters because the paper's explanation of stagnation and the motivation for PA-AA ('breaking the stagnation barrier') depend on the iteration being trapped, not merely moving slowly. The concern is sharpened by Table 1, where the same HINTS construction reaches residual ~1e-5 after 700 cycles at N=201; at N=801 the plateau may simply be a much longer transient. Without a convergence-floor test, the 'false fixed point' label is not established, and the advantage of PA-AA over simply running longer is unclear.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that the reliability of deep-learning-based hybrid iterative methods (DL-HIMs) for PDEs depends not only on neural architecture but also on training paradigms and update strategies. It formalizes a HINTS-type cycle as a fixed-point iteration and claims that a spectral gap between the classical smoother and the neural operator can produce 'false fixed points' where the full-cycle update vanishes while the physical residual remains large. It compares static and dynamic training, error- and residual-based losses, and several update strategies, including a new physics-aware Anderson acceleration (PA-AA) that minimizes the physical residual instead of the fixed-point update. Numerical experiments on one-dimensional diffusion and Helmholtz problems suggest that residual-based losses help DeepONet-based HINTS, that static residual losses can destabilize FNS, and that PA-AA breaks stagnation and reaches much smaller residuals than standard Anderson acceleration.","tokens_in":10828,"tokens_out":7949,"duration_ms":76879,"significance":"If the claims are fully substantiated, the paper makes a useful contribution to an active debate: it shifts attention from architecture design to the joint design of training objectives and iteration strategies, and it offers a simple, practical acceleration variant. The explicit comparison of two structurally different neural operators and the public code repository are strengths. However, the empirical support is currently narrower than the claims: the central false-fixed-point mechanism is inferred from a single plateau, no multiple seeds or error bars are reported, and the PA-AA advantage is demonstrated on single trajectories without a convergence theory or cost analysis. The paper is therefore promising but not yet conclusive.","major_comments":[{"comment":"The false-fixed-point diagnosis is not established. The update norm plateaus near 1e-4, not zero, and no long-horizon run is shown. For the same HINTS construction at N=201, Table 1 reports a final residual of 6.9e-5 after 700 cycles, so the N=801 plateau may be a long transient rather than a true fixed point. Since the motivation for PA-AA and the claim that standard AA is ill-suited both rest on the iteration being trapped, the authors should (i) run the unaccelerated iteration to a convergence floor over a much longer horizon, (ii) report multiple independent random-field instances/seeds, and (iii) if possible verify numerically that G(u_*) is indistinguishable from u_* while f - A u_* remains large. If the residual is still decreasing, the phenomenon should be relabeled slow convergence rather than a false fixed point.","section":"Section 5.2, Figure 1"},{"comment":"The central PA-AA claim, 'converges robustly to 1e-9', is based on a single trajectory plot with no stopping criterion, no seeds, and no error bars. Because PA-AA minimizes ||f - A u||_2, the reported residual is the same objective being optimized; for an indefinite Helmholtz problem this may overstate accuracy, so solution-error curves are needed as an independent metric. The paper also does not report wall-clock time or per-iteration cost; PA-AA adds a least-squares solve and additional residual evaluations each cycle, so 'fewer iterations' does not yet demonstrate lower cost. Please report statistics over the test set, define a convergence threshold, include error curves, give a cost comparison in time or FLOPs, and state clearly whether PA-AA is heuristic or has convergence guarantees.","section":"Section 5.5, Figure 4 and Conclusion"},{"comment":"The cost-efficiency conclusion ('dynamic training should be adopted with caution') rests on a single comparison with K=5. The 8.63x training-time and 1.27x memory figures are reported without measurement methodology or variance, and the final errors/residuals differ by only a small factor (3.4e-7 vs 4.4e-7; 6.9e-5 vs 1.0e-4). These results do not eliminate the possibility that a larger unrolling horizon K would substantively improve convergence, nor do they establish that the static model's advantage is significant. The authors should either report multiple independent training runs with standard deviations or weaken the conclusion to a statement about this specific configuration.","section":"Section 5.4, Table 1"},{"comment":"The claim that residual-based objectives are architecture-dependent is supported by only a single experiment per architecture, all in 1D. The FNS 'diverge' curves are not quantified (divergence threshold? blow-up or slow growth?), and no seeds are reported. Given the broad title and the conclusion about general DL-HIM reliability, the conclusions need either more systematic variation (problem dimension, n, wavenumber, loss norm, lambda, network initialization) or an explicit restriction of the claims to the tested 1D settings.","section":"Section 5.3, Figure 3 and Section 5.5"}],"minor_comments":[{"comment":"The paper is not consistently formatted; the opening sentence of Section 1 has missing spaces between words, and the Conclusion contains oddly capitalized headings such as 'T raining Objectives'. A careful copyedit is needed.","section":"General"},{"comment":"The notation 'δ(k) = 0⇐ ⇒r(k) = 0' is typeset confusingly; use a proper double implication symbol.","section":"Eq. (5)"},{"comment":"PA-AA is presented as new, but its structure is a residual-minimizing Anderson type. The discussion should relate PA-AA to existing nonlinear Krylov/Anderson variants and state precisely what is new.","section":"Algorithm 2"},{"comment":"When the adaptive step size is used for non-SPD problems, the denominator p_k^T A p_k may vanish or change sign. The text says this is treated heuristically, but the experiments should report how such cases were handled numerically.","section":"Section 4.1"},{"comment":"The choice n=19 Jacobi steps per cycle is fixed throughout. A sensitivity test with respect to n would help determine whether the observed stagnation is robust or an artifact of this specific cycle split.","section":"Section 5.1"},{"comment":"Training hyperparameters such as network widths, training epochs, optimizer, learning rate, and the H1 loss weight lambda are not reported in the text. The code repository helps, but the manuscript should state these for reproducibility.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the comparison framework is useful, but the evidence is too thin in several places that the abstract treats as established. The unverified false-fixed-point diagnosis and the absence of multiple seeds are the main risks. I would ask for the additional experiments described in the major comments rather than reject, because the central idea is defensible and the code availability is a positive contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this paper makes a genuinely useful empirical case that how you train and update a hybrid AI-classical PDE solver matters as much as the network architecture. The comparison of static vs. dynamic training, error- vs residual-based losses, and the proposed PA-AA variant is the kind of systematic study the DL-HIM field needs. The result that residual-based losses help DeepONet-based HINTS but can destabilize FNS under static training is a concrete, non-obvious finding. PA-AA, which minimizes the physical residual rather than the fixed-point update, is a small but sensible modification, and the experiments show it breaks stagnation that standard AA does not. Credit where due: the code is public, and the comparison against standard AA with the same memory size gives independent evidence that the residual-minimizing combination is doing real work, not just re-reporting the training loss.\n\nThe soft spots are real but not fatal. The headline mechanism—'false fixed points' where the update vanishes while the residual stays large—is inferred from a plateau at update norm ~1e-4 with residual ~1e1. That could indeed be a true fixed point of the iteration, but it could also be a very slow transient. Table 1 shows the same HINTS setup reaching 1e-5 residual after 700 cycles at N=201; at N=801 the plateau may just be a much longer transient. A longer-horizon run or a convergence floor test would settle it. This matters because the paper's motivation for PA-AA is 'breaking the stagnation barrier,' and if the barrier is just slow convergence, the advantage of PA-AA over running longer is less dramatic. Minor issues: no error bars or multiple seeds, the PA-AA memory size is chosen post hoc, and the conclusions are broader than two 1D problems warrant. The dynamic training cost analysis is honest—8.63x training time for no gain—and the authors do not oversell it.\n\nWho this is for: anyone working on neural PDE solvers, especially hybrid iterative methods. It will not reshape the field, but it gives practical guidance and a new acceleration trick worth trying. I would send it to peer review, with the expectation that the authors tighten the false-fixed-point diagnosis and scale back the generality claims. I'd cite it if I worked on DL-HIMs, and I'd probably bring it to a reading group focused on scientific ML.","headline":"Plausible, useful empirical study of DL-HIM training/update strategies, but the central 'false fixed point' claim needs a longer-horizon check before it carries the weight the paper puts on it.","tokens_in":11280,"tokens_out":1042,"would_cite":true,"duration_ms":11954,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65F10","65N22","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Deep learning-based hybrid PDE solvers can stall at false fixed points where updates vanish but residuals stay large; the paper shows that aligning training objectives and using residual-based acceleration restores reliable convergence.","keywords":["deep learning-based hybrid iterative methods","false fixed points","Anderson acceleration","neural operators","Helmholtz equation","training objectives","physics-aware acceleration","convergence stagnation"],"falsifier":"Run the HINTS iteration from Figure 1 for a much longer horizon (e.g., 10^5 cycles) and track the physical residual; if it eventually drops below, say, 1e-2, the plateau is slow convergence rather than a fixed point, which would weaken the mechanistic motivation for PA-AA.","tokens_in":10430,"feed_emoji":"🧮","tokens_out":6256,"duration_ms":57980,"temperature":0.7,"pith_summary":"The paper asks whether deep learning-based hybrid iterative solvers (DL-HIMs), which alternate classical smoothers with neural operators, can be trusted to solve PDEs. It argues that their frequent stagnation arises not from network architecture but from two design choices: the training objective and the update rule. Through experiments on a DeepONet-based solver (HINTS) and an FFT-based solver (FNS), it shows that residual-based losses work best for the former while error-based losses are safer for the latter under static training, and that fixed step sizes or standard Anderson acceleration can push a nonlinear solver toward a false fixed point. The paper introduces physics-aware Anderson acceleration (PA-AA), which minimizes the physical residual rather than the update magnitude, and demonstrates that it lets HINTS converge to residuals around 1e-9 on Helmholtz problems where standard acceleration stalls above 1e-3. If correct, this reframes the reliability debate: hybrid solvers are not inherently unreliable, but demand physically informed training and iteration design.","feed_headline":"Residual-aware acceleration fixes stalled hybrid PDE solvers","feed_subtitle":"A tweak to the update rule lets nonlinear learned solvers reach residuals below 1e-9 on Helmholtz problems.","key_machinery":"The key objects are the DL-HIM fixed-point map G_theta = M_{N_theta} composed with M_S^n (a cycle of n smoother steps followed by one neural correction) and the false-fixed-point phenomenon where delta_k = G_theta(u_k) - u_k tends to 0 while r_k = f - A u_k stays large. The main new mechanism is physics-aware Anderson acceleration (PA-AA), a multisecant acceleration that solves the constrained least-squares problem for coefficients alpha_j minimizing ||sum alpha_j (f - A g_{k-j})|| instead of ||sum alpha_j (g_{k-j} - u_{k-j})||, thereby forcing the accelerated iterate to reduce the PDE residual directly.","core_discovery":"The central claim is that a DL-HIM is a fixed-point iteration u^{k+1} = G_theta(u^k) whose mathematical fixed points need not coincide with the PDE solution. Because the classical smoother and the neural operator have complementary spectral biases, there is a spectral gap: on fine grids the smoother barely moves low-frequency error, and the neural network may fail to correct certain modes, making the full-cycle update tiny while the physical residual r = f - A u remains large. This 'false fixed point' explains the stagnation plateau observed in HINTS. The paper substantiates this with a systematic comparison of training objectives and update strategies, and shows that PA-AA—which replaces th","pith_inferences":["The false-fixed-point diagnosis suggests a general lesson: any learned preconditioner or iterator used inside a fixed-point loop should be certified by residual monitoring, not by update norm alone; this could apply to other learned iterative schemes beyond DL-HIMs.","Because PA-AA only needs the operator A and the residual f - A u, it is a drop-in acceleration for any nonlinear fixed-point iterator; testing it on classical nonlinear solvers (e.g., domain-decomposition or multigrid with inexact coarse solves) that exhibit stagnation would be a natural next step.","The paper compares static and dynamic training only for HINTS with a short horizon K=5; one could test whether longer unrolling horizons or finer target grids make dynamic training competitive, which the current cost analysis leaves open.","The observed preference of FNS for error-based losses under static training hints that the architecture's spectral operating mode changes how residual-based losses stress the network; a frequency-domain analysis of the loss landscape might predict such preferences a priori."],"forward_implications":["Practitioners should monitor the physical residual rather than the update norm when judging whether a hybrid solver has converged.","Training objectives should be chosen alongside the architecture: residual-based losses are a better default for DeepONet-based hybrids, while FFT-based solvers like FNS may need error-based objectives if trained statically.","The cost of dynamic training (unrolling K iterations) may not be justified: in these tests K=5 increased training time by about 8.6 times and memory by 1.27 times with little convergence benefit.","Standard Anderson acceleration can be counterproductive for nonlinear neural operators; a residual-aware variant (PA-AA) accelerates both linear and nonlinear solvers to much smaller residuals in fewer iterations.","The results give a concrete answer to the reliability debate: DL-HIMs are not inherently unreliable, but require coherent design of architecture, training objective, and update strategy."],"fun_headline_variants":["False fixed points explain stalled neural PDE solvers","Physics-aware Anderson acceleration fixes hybrid solver plateaus","Training paradigm, not architecture, decides hybrid solver reliability","Anderson acceleration needs physics awareness for neural solvers","Hybrid PDE solvers stagnate at false fixed points"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the plateau in Figure 1—where the update norm is about 1e-4 but the residual is about 1e1—is a genuine false fixed point, not merely slow convergence; the paper never tests whether the iteration would escape the plateau with longer runs or different tolerances.","fun_headline_variants_meta":{"raw":{"variants":["False fixed points explain stalled neural PDE solvers","Physics-aware Anderson acceleration fixes hybrid solver plateaus","Training paradigm, not architecture, decides hybrid solver reliability","Anderson acceleration needs physics awareness for neural solvers","Hybrid PDE solvers stagnate at false fixed points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000464,"raw_usage":{"total_tokens":2156,"prompt_tokens":744,"completion_tokens":1412,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":1338}},"tokens_in":488,"tokens_out":1412,"duration_ms":9867,"temperature":1.0,"reasoning_tokens":1338,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T03:45:05.190766+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the HINTS iteration from Figure 1 for a much longer horizon (e.g., 10^5 cycles) and track the physical residual; if it eventually drops below, say, 1e-2, the plateau is slow convergence rather than a fixed point, which would weaken the mechanistic motivation for PA-AA.","supporting_citations":[],"review_version":1}