{"id":"0ba6150d-4675-4b7c-9fed-0512b51b3107","arxiv_id":"2502.06865","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Adding Fourier features to the input layer lets the Deep Ritz Method generate high-frequency solutions for non-convex multi-well energy problems, while increasing network depth alone does not consistently do so.","lead":"This paper combines Fourier feature mapping with the Deep Ritz Method so neural networks can produce high-frequency, oscillatory solutions for non-convex energy minimization problems. The approach is a mesh-free route to approximating microstructure patterns in materials like shape-memory alloys.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No energy values are reported for the no-minimizer benchmarks (17)/(18), so high-frequency plots and transition counts do not establish that DRM&FM produces minimizing sequences; the central 'solving' claim is unsupported.","rationale":"I read the paper in good faith. The empirical strategy is reasonable: for problems without minimizers, visualizing oscillations is a natural first step, and the Fourier-feature mechanism is plausible. However, the abstract's 'solving variational models' and 'generate solutions' claims require that the displayed functions actually have energy approaching the infimum. The paper never shows this; it counts transitions. The reader's weakest assumption identifies exactly this gap. My additional checks confirm it: Theorem 1 assumes strict convexity, which none of the Lagrangians in (15)-(18) satisfy, and the paper explicitly disclaims a theoretical accuracy guarantee; the depth-independence claim has a counterexample in Fig. 4(c). These are not accusations, and there is no internal inconsistency in the raw numerical results as presented — simply missing evidence. The appropriate verdict is unchanged: CONDITIONAL, pending energy reporting and ideally a few random restarts. A single dense-quadrature energy audit would settle whether the visual high-frequency behavior is also variational convergence.","tokens_in":15373,"tokens_out":5290,"duration_ms":58457,"concrete_test":"Run the three benchmarks with the Section 4 hyperparameters and, at every checkpoint and for each depth/frequency, compute I(u-hat) by dense quadrature (e.g., 10^5 uniform points in 1D; a 100x100 grid in 2D), reporting the separate terms ∫(u_x^2-1)^2, ∫u^2, and the boundary penalty, along with the final transition counts. Plot log I vs epoch. The concern lands if the high-frequency DRM&FM solutions do not have substantially lower I than the no-Fourier baseline at comparable depth, or if their energies are not orders of magnitude closer to the known infimum 0; if the high-frequency plots have large derivative-error energy, they are not minimizing sequences.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Fourier feature mapping allows DRM to generate high-frequency, multiscale solutions to the non-convex benchmarks. For (17) and (18), the infimum is 0 but no minimizer exists; the only quantitative evidence offered is the number of transitions between u_x=±1 or u_y=±1 seen in Figures 5-8. No values of I(u-hat) are reported at any epoch or final state. A function with many transitions can have large energy if its derivative spends significant measure away from the wells or if the lower-order term is large; under u(0)=u(1)=0, a sawtooth with slopes ±1 still carries nonzero ∫u^2. Thus transition count alone does not measure proximity to the infimum. The theory cannot close this gap: Theorem 1 assumes W strictly convex in (u,u') and positive definite D_X, whereas the benchmarks are non-convex with indefinite Hessians, as the authors concede in the remark after Theorem 1 and in Section 5's statement that no theoretical foundation quantifies solution accuracy. The depth-independence assertion is also weakened by Fig. 4(c), where 5- and 7-layer nets give 32 transitions but the 9-layer net gives 16, and Section 4.2 admits the frequency/transition relationship was not investigated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes combining the Deep Ritz Method (DRM) with Fourier feature mapping (DRM&FM) to solve non-convex variational problems arising in microstructure modeling. The authors first present a neural tangent kernel (NTK) analysis for DRM, arguing that DRM suffers from spectral bias because convergence along different directions is controlled by the eigenvalues of the NTK Gram matrix. They then argue that Fourier feature mapping, by mapping inputs onto the unit circle, can lead to a quadratically decaying NTK spectrum, which may alleviate spectral bias. Numerically, the paper studies three benchmarks: a 1D double-well problem (15) with attained minimizers, a 1D double-well problem with a lower-order term (17) whose infimum is zero but is not attained, and a 2D twin-branching problem (18) also without an attained minimizer. For each problem, the authors show derivative plots and count transitions between preferred states, comparing DRM with and without Fourier features at various network depths and frequencies. The central claim is that DRM&FM generates high-frequency, multiscale minimizing sequences independently of network depth, while plain DRM does not.","tokens_in":15674,"tokens_out":2244,"duration_ms":23747,"significance":"If the central claim is validated, the paper offers a practical, mesh-free alternative to finite elements for a difficult class of non-convex, microstructure-scale variational problems, and it provides a useful NTK-based heuristic for why Fourier features help DRM. The concrete NTK derivation for DRM in Section 3.1, with the explicit loss dynamics (14) and the spectral decomposition in Theorem 1, is a valuable contribution in itself, even though its assumptions are not met by the benchmarks. The numerical experiments cover both 1D and 2D settings and include a regularization study in Section 4.4, which strengthens the empirical picture. However, the evidence supporting the main 'solving' claim is incomplete because no energy values are reported for the no-minimizer benchmarks, and the depth-independence claim is qualified by a counterexample in Figure 4(c). The paper is therefore of interest to the community, but its central numerical claim requires additional verification before it can be accepted as stated.","major_comments":[{"comment":"The central claim that DRM&FM produces minimizing sequences for the no-minimizer benchmarks is not supported by the reported evidence. For problems (17) and (18), the infimum is zero but is not attained, so a function with many transitions can still have energy arbitrarily far from the infimum if its derivative wanders away from the wells or if the lower-order term (for (17)) contributes significantly. The paper reports only transition counts and derivative plots (Figures 5-10) without ever reporting the computed energy I(û) at any epoch or at convergence. I strongly recommend adding energy trajectories or final energy values, and ideally a comparison with the known infimum, for all benchmarks and for both DRM and DRM&FM at the frequencies and depths tested. Without this, the assertion that DRM&FM 'generates minimizing sequences' is not quantitatively supported.","section":"Sections 4.2 and 4.3, Eqs. (17) and (18)"},{"comment":"The abstract and Sections 1 and 4 claim that Fourier feature mapping enables high-frequency solutions 'independently of the depth of the DNN' or 'independently of the depth' (Section 1). Figure 4(c) directly contradicts this for benchmark (15): with Fourier frequency i=4, the 5- and 7-layer networks produce 32 transitions, but the 9-layer network produces 16 transitions. The text in Section 4.1 also states that 'one cannot quantify the relationship between mapping frequency and number of transitions within the domain,' and Section 4.2 admits the frequency-transition relationship was not investigated. The depth-independence claim should either be restricted to the frequencies and depths where it actually holds, or the authors should provide a more systematic study that quantifies the dependence on depth, frequency, and training epochs. As written, the claim is too strong relative to the presented evidence.","section":"Figure 4(c) and Section 4.1"},{"comment":"Theorem 1 requires W to be strictly convex in (u, u') and the Hessian blocks D_X to be positive definite, as stated in the theorem assumptions. The benchmarks studied in Section 4 are all non-convex, and the authors themselves note in the remark following Theorem 1 that 'for a non-convex Lagrangian W, the convergence of ∇U ¯W_X requires a more refined analysis.' This means the NTK-based spectral-bias explanation is rigorously valid only for a different class of problems than the ones actually solved. The paper should be explicit that Theorem 1 is a heuristic motivation, not a proof of the observed behavior for the non-convex benchmarks. Alternatively, the authors could include a finite-width or direct eigenvalue analysis for the specific non-convex W used in the experiments, or at least empirically examine the decay of the NTK eigenvalues for those W.","section":"Section 3.1, Theorem 1"}],"minor_comments":[{"comment":"The optimizer is referred to as 'Adams Optimizer' in Sections 4.1, 4.2, and 4.3; the standard name is 'Adam' (Kingma and Ba, reference [26]). Please correct this typo throughout.","section":"Section 2"},{"comment":"The caption of Figure 5 says 'First Row (a)-(c): DRM approximation to (17) with ReLU activation function, ε = 0, η = 1.0 × 10⁻⁴' and 'Second Row (d)-(f)', but the notation 'ε = 0' is unexplained in the 1D context; the epsilon parameter is defined only for the 2D regularized problem (19). Please clarify what ε refers to in Figures 5 and 6.","section":"Figure 5 caption"},{"comment":"The caption of Figure 4 states that the Fourier mapping is 'of frequency δ(x) = (sin(2iπx), cos(2iπx))' but the frequency is 2iπ, not i; the text in Section 4.1 says 'a Fourier mapping of frequency 4π' for i=2, which is consistent. Please make the caption consistent with the text to avoid confusion.","section":"Section 4.1, Figure 4 caption"},{"comment":"Appendix B states that the eigenvalues of the Gram matrix M_X are approximately λ_k ≈ nΛ_k for large n, and the main text in Section 3.2 uses the 'quadratic decay' of the NTK eigenvalues from reference [23]. However, the connection between the decay rate of the integral operator eigenvalues and the decay rate of the Gram matrix eigenvalues for finite n is only sketched. A brief comment on the finite-n regime would help the reader gauge how directly the O(k^{-2}) result applies to the numerical experiments with n = 128 or n = 1000.","section":"Appendix B"},{"comment":"The concluding sentence acknowledges that the method 'lacks a theoretical foundation to quantify solution accuracy for the considered minimization problems.' This is honest and appropriate, but given that the benchmarks are exactly of that type, the paper should also present empirical energy values as the substitute form of accuracy verification, which is currently missing (see major comment 1).","section":"Section 5, conclusions"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and timely problem, and the NTK derivation for DRM is a contribution that could be of interest to the journal. My main concern is evidentiary: the paper's headline claim is that DRM&FM 'solves' non-convex variational problems, but for the two benchmarks where the infimum is zero and unattained, only transition counts are reported, not energy values. This is not a matter of taste; a transition count is not a convergent quantity for these problems, and the paper does not show that the plotted functions actually form a minimizing sequence. The depth-independence claim is also contradicted by the 9-layer case in Figure 4(c). These issues are fixable within the manuscript's scope by adding energy trajectories, a more systematic parameter study, and a more carefully qualified abstract. I therefore recommend major revision rather than rejection. I would also encourage the editor to consider whether the paper's claims about microstructure 'solving' require a more standard benchmark comparison (e.g., with FEM or quasiconvexification) to fully substantiate the practical significance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe short version: this is a plausible, honestly written paper that shows Fourier feature mapping can push Deep Ritz solutions toward more high-frequency oscillatory structure on three non-convex toy benchmarks. What is actually new is the specific combination—Fourier features applied to DRM, with a DRM-specific NTK derivation in Theorem 1—and the observation that this helps generate more twin bands without increasing network depth. The paper gives a clear account of why DRM alone suffers spectral bias, and the numerical figures support the claim that Fourier mapping increases transition counts.\n\nThe soft spots are real. For problems (17) and (18), which have no minimizer and infimum zero, the authors report no energy values at any epoch. The only evidence is the number of transitions between preferred states in the plots. A function with many transitions can still have significant energy if the derivative wanders away from the wells, and under zero boundary conditions the lower-order term alone prevents energy from approaching zero. So the transition count does not demonstrate proximity to the infimum. The paper would be much stronger with a plot of I(û) over training, or a table of final energies. That is a load-bearing omission for the claim that DRM&FM 'solves' these problems.\n\nThe theory is also narrower than the applications. Theorem 1 assumes W strictly convex in (u,u') and positive definite D_X, whereas the benchmarks are non-convex with indefinite Hessians. The authors concede this in the remark after Theorem 1 and again in the conclusion, where they state there is no theoretical foundation to quantify solution accuracy. That honesty is good, but it means the theoretical section only motivates the method; it does not explain the non-convex results.\n\nOne more specific issue: the depth-independence claim is too strong. In Figure 4(c), 5- and 7-layer nets give 32 transitions but the 9-layer net gives 16. The authors mention this but still summarize the behavior as 'independently of the depth'. A more careful wording would be 'less dependent on depth' or 'depth is not the controlling factor'.\n\nOverall, this is a useful contribution for researchers working on mesh-free methods for non-convex variational problems. It is a natural combination of known components rather than a fundamental breakthrough, but the question—can we control the frequency content of DRM solutions without mesh refinement?—is relevant and the paper gives a reasonable empirical start. It deserves a serious referee, especially if the referee asks for energy values and a revised depth-independence claim. I would not cite it in my own work yet, but I might bring it to a reading group.","headline":"Fourier features help DRM produce more high-frequency structure on non-convex benchmarks, but missing energy values for the no-minimizer problems leave the 'solving' claim only partially supported.","tokens_in":16153,"tokens_out":2226,"would_cite":false,"duration_ms":21652,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fourier features give Deep Ritz fine-scale solutions without deeper networks.","keywords":["deep learning","variational problems","nonconvex energy minimization","Fourier feature mapping","martensitic phase transformation","Deep Ritz method","neural tangent kernel","spectral bias"],"falsifier":"Compute $I(u)$ for the DRM&FM outputs of problems (17) and (18) and test whether it approaches zero as Fourier frequency or training epochs increase; if the energy stays bounded away from zero, the visible oscillations are not minimizing sequences.","tokens_in":15203,"feed_emoji":"🔬","tokens_out":11088,"duration_ms":95001,"temperature":0.7,"pith_summary":"This paper argues that the Deep Ritz Method (DRM), a neural-network approach that solves variational problems by minimizing an energy functional directly, is biased toward low-frequency solutions, and that applying a Fourier feature mapping to the input layer removes this bias. If correct, the result matters for materials science: microstructures that form when a material changes crystal structure are high-frequency, multiscale patterns minimizing non-convex energies, and mesh-based methods struggle to resolve them. The paper derives a neural-tangent-kernel (NTK) theorem showing that DRM's convergence rate is controlled by NTK eigenvalues, and argues that Fourier features flatten that eigenvalue spectrum. Numerically, it shows that DRM with Fourier features generates high-frequency minimizing sequences for three benchmark problems (two 1D, one 2D) independently of network depth.","feed_headline":"Fourier features let Deep Ritz resolve high-frequency microstructure","feed_subtitle":"Deep Ritz normally learns low frequencies; adding Fourier features yields multiscale minimizing sequences at fixed depth.","key_machinery":"The load-bearing object is the DRM neural tangent kernel, the matrix-valued kernel $K(x_m,x_n;\\theta)=\\nabla_\\theta U_m(\\theta)[\\nabla_\\theta U_n(\\theta)]^T$, where $U_n=(u(x_n),u'(x_n))^T$ stacks the network output and its spatial derivative. This kernel enters a Gram matrix $M_X$ whose spectral decomposition $D_X M_X = Q\\Lambda Q^T$ sets the per-direction convergence rates in Theorem 1. The second ingredient is the Fourier feature map; by sending the input to the unit circle $S^1$ (or adjoining $x$ in 2D), it makes the NTK eigenvalue decay quadratic in the eigen-index, which is what allows the method to learn high frequencies.","core_discovery":"On its own terms, the central claim is that DRM inherits the spectral-bias pathology of neural networks: the gradient-descent dynamics of the loss decompose into eigen-directions of a DRM-specific NTK, and directions with small eigenvalues converge exponentially slowly. Theorem 1 states that under lazy training, convexity of the Lagrangian in $(u,u')$, and positive definiteness of the Gram matrix, the component of the gradient along eigen-direction $q_i$ decays as $e^{-\\eta \\lambda_i t/|X|}$. Fourier feature mapping $\\delta(x) = (\\sin(2^i \\pi x), \\cos(2^i \\pi x))$ in 1D (and $(x, \\sin(2^i \\pi x), \\cos(2^i \\pi x))$ in 2D) maps the input onto a submanifold on which the NTK eigenspectrum has quadratic decay, which the paper takes to homogenize learning across scales. The numerical section demonstrates that with this mapping, deeper networks are not needed: the number of transitions between preferred states (and twin bands in 2D) grows with the Fourier frequency, and a regularization term controls smoothness and interface spacing.","pith_inferences":["A testable extension is to report the energy $I(u)$ of the outputs for benchmarks (17) and (18) and check whether it approaches the infimum; the paper's evidence is transition counts in plots, so an energy-based check would directly settle whether the outputs are minimizing sequences.","The observed tie between Fourier frequency and transition count suggests an adaptive frequency schedule, where the input-map frequencies are annealed or selected from the current energy landscape instead of fixed at $i=1,\\dots,4$.","If the quadratic-decay NTK result extends beyond the 1D $W=W(x,u)$ case, the same input mapping may transfer to other non-convex variational problems with oscillatory minimizers, such as micromagnetics or phase-field models.","The paper leaves the non-convex version of Theorem 1 open; a rigorous treatment would need to replace the convexity and lazy-training assumptions with a variational analysis of the non-convex landscape, which the conclusions identify as future work."],"forward_implications":["For the 1D double-well problem (15), plain DRM with 5, 7, and 9 hidden layers produces only one transition between the two preferred slopes, while DRM&FM with Fourier frequencies $4\\pi$ and $8\\pi$ produces 4 and 8 transitions at the same depth.","For the non-attaining 1D benchmark (17), DRM&FM at frequency $2\\pi$ produces about 12 transitions with a fixed 3-hidden-layer network, rivaling a 7-hidden-layer plain DRM; higher Fourier frequencies increase the transition count.","For the 2D twin-branching problem (18), Fourier frequencies $2\\pi,4\\pi,8\\pi,16\\pi$ produce 4, 8, 15, and 32 twin bands, whereas plain DRM with 7 hidden layers stalls at an unstable state.","Adding the regularization term $\\varepsilon^2 u_{yy}^2$ smooths the minimizing sequences and limits the number of interfaces; larger $\\varepsilon$ yields lower-frequency solutions.","The NTK analysis identifies the eigenvalue spectrum of the DRM kernel as the mechanism behind spectral bias, so Fourier features target the convergence-rate bottleneck directly rather than relying on network depth."],"supporting_citations":[{"why":"Baseline DRM treatment of similar non-convex benchmark problems and the depth-as-mesh hypothesis this paper challenges.","marker":"[21]"},{"why":"Introduces Fourier feature mapping and demonstrates that it lets networks learn high-frequency functions in low-dimensional domains.","marker":"[22]"},{"why":"Supplies the theorem that the NTK restricted to S^1 has quadratic eigenvalue decay, the theoretical basis for the Fourier-feature claim.","marker":"[23]"},{"why":"Supports the equivalence between deep NTK and Laplace-kernel reproducing-kernel Hilbert spaces used in the Fourier-feature argument.","marker":"[24]"},{"why":"Documents the spectral bias pathology of deep networks that the paper extends to DRM.","marker":"[19]"},{"why":"Shows eigenvector-bias and Fourier-feature behavior for multi-scale PDEs, motivating the DRM analysis.","marker":"[20]"},{"why":"Characterizes minimizers of the double-well benchmark and the non-attaining nature of the related problems.","marker":"[4]"},{"why":"Supplies the lazy-training assumption on which the DRM NTK theorem's linearized dynamics rest.","marker":"[29]"},{"why":"Original Deep Ritz formulation that defines the energy loss and boundary-penalty term minimized here.","marker":"[25]"}],"fun_headline_variants":["Fourier mapping fixes spectral bias in Deep Ritz","Deep Ritz multiscale via Fourier feature mapping","Fourier features overcome DRM spectral bias","Spectral bias cured with Fourier features in Deep Ritz","Deep Ritz gains high-frequency power via Fourier features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a network output whose plot shows many transitions between preferred states is a minimizing sequence; for the benchmarks with no true minimizer the paper does not report the energy values that would show approach to the infimum.","fun_headline_variants_meta":{"raw":{"variants":["Fourier mapping fixes spectral bias in Deep Ritz","Deep Ritz multiscale via Fourier feature mapping","Fourier features overcome DRM spectral bias","Spectral bias cured with Fourier features in Deep Ritz","Deep Ritz gains high-frequency power via Fourier features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2903,"prompt_tokens":935,"completion_tokens":1968,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":1891}},"tokens_in":551,"tokens_out":1968,"duration_ms":14667,"temperature":1.0,"reasoning_tokens":1891,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T19:25:41.308499+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute $I(u)$ for the DRM&FM outputs of problems (17) and (18) and test whether it approaches zero as Fourier frequency or training epochs increase; if the energy stays bounded away from zero, the visible oscillations are not minimizing sequences.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline DRM treatment of similar non-convex benchmark problems and the depth-as-mesh hypothesis this paper challenges."},{"cited_title":"Tancik, P","cited_arxiv_id":null,"evidence_quote":"Introduces Fourier feature mapping and demonstrates that it lets networks learn high-frequency functions in low-dimensional domains."},{"cited_title":"Geifman, A","cited_arxiv_id":null,"evidence_quote":"Supplies the theorem that the NTK restricted to S^1 has quadratic eigenvalue decay, the theoretical basis for the Fourier-feature claim."},{"cited_title":"Rahaman, A","cited_arxiv_id":null,"evidence_quote":"Documents the spectral bias pathology of deep networks that the paper extends to DRM."},{"cited_title":"Carstensen, Ten remarks on nonconvex minimisation for phase transition simulations, Computer Methods in Applied Mechanics and Engineering 194 (2) (2005) 169–193","cited_arxiv_id":null,"evidence_quote":"Characterizes minimizers of the double-well benchmark and the non-attaining nature of the related problems."},{"cited_title":"Chizat, E","cited_arxiv_id":null,"evidence_quote":"Supplies the lazy-training assumption on which the DRM NTK theorem's linearized dynamics rest."},{"cited_title":"Weinan, B","cited_arxiv_id":null,"evidence_quote":"Original Deep Ritz formulation that defines the energy loss and boundary-penalty term minimized here."}],"review_version":1}