{"id":"9a29ef8e-b795-4ffc-b5be-03294a6d6051","arxiv_id":"2607.28762","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adding NFM-style bi-interaction layers to PINNs and DeepONets improves accuracy on several high-dimensional smooth PDEs and shock-dominated conservation laws, but not on low-dimensional smooth problems.","lead":"The paper plugs factorization-machine-style pairwise feature interactions into PINNs and neural operators, and reports accuracy gains on high-dimensional smooth PDEs and on shock-dominated operator benchmarks. A generalist should care because it offers a simple, parameter-efficient architectural change in a very active area of AI-for-science.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing no-interaction control for FM-Operator: gains may come from the new embedding+MLP architecture, not the bi-interaction pooling, leaving the central causal claim under-supported.","rationale":"The paper is an honest empirical study of a plausible architectural idea, and the operator-half evidence is substantially stronger than the PINN-half evidence: multi-seed runs, parameter-matched baselines, and a credible Shift-DeepONet comparison. The reader's identified weakest assumption — benchmark representativeness, reinforced by the Appendix D reversal between separable sine and quadratic-phase manufactured solutions — is a valid external-validity concern. I treat it as secondary. The more load-bearing issue is causal attribution: what would have to be true for the central claim is that explicit pairwise interactions, not the surrounding redesigned architecture, drive the observed gains. For FM-Operator, the paper never tests this directly. The split/non-split ablation keeps the interaction block in both arms; the stats-only control changes only the input features; short DeepONet changes the baseline architecture. FM-DeepONet is closer to a clean test but is not the model carrying the main headline claims in the abstract. The paper's own Section 4 disclaimer (inductive bias, not universal approximation) and the mixed results across benchmarks show the authors are aware of limits, which is good faith, but the Discussion's attribution is stronger than the evidence. A single no-interaction control on the main sharp-gradient benchmarks would settle whether the interaction module is necessary or whether the gains are a property of the new embedding/decoder design. That is an addressable condition, so it does not change the reader's CONDITIONAL verdict: the paper should be accepted only after such a control is added.","tokens_in":33165,"tokens_out":12120,"duration_ms":131310,"concrete_test":"Run a no-interaction FM-Operator control on the five seeded sharp-gradient benchmarks (parametric Burgers, cubic conservation, LWR, periodic Burgers, square-pulse advection): replace z_BI with the elementwise mean (or a concatenation followed by a single linear projection to the same 128-d input) of the same per-field embeddings, keep the linear first-order path, seeds, WENO5/finite-volume references, 30k-step schedule, and checkpoint rule identical, and match parameter count as closely as possible. If the control's mean relative L2 stays within ~10% of FM-Operator on most benchmarks or still beats DeepONet by the reported margins, the interaction-specific explanation fails; if the control loses most of the gap, the bi-interaction pooling is confirmed as the active ingredient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that factorization-machine-style pairwise interactions are the mechanism behind the accuracy gains. FM-DeepONet supplies a partial control: the interaction block is the main addition to a standard DeepONet. But the headline FM-Operator results compare a completely redesigned architecture — per-field affine embeddings, bi-interaction pooling, an MLP decoder, and an explicit linear first-order path — against DeepONet. The ablations reported (short DeepONet, compact FM-DeepONet, DeepONet+stats, split vs non-split query) rule out parameter count, the three sensor statistics, and query grouping as explanations; they do not rule out the embedding+MLP decoder/linear-path design itself. Section 4 candidly says the block is an inductive bias, not a universal-approximation gain, and Section 6 attributes the gains to 'how the models construct cross-field interactions,' but no experiment removes the interaction pooling from FM-Operator while keeping the rest of the architecture. If a non-interacting version (e.g., elementwise sum or concatenation of the same embeddings) performs as well, then explicit cross-field products are not load-bearing; the contribution would reduce to a generic feature-embedding MLP. That would undercut the paper's title and central abstraction, independent of benchmark representativeness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes embedding factorization-machine-style bi-interaction layers into PINNs and neural operators. FM-PINN treats each coordinate as a separate field and computes pairwise Hadamard products of field embeddings before an MLP decoder; FM-Operator applies the same interaction pooling to sensor values, query coordinates, sensor statistics, and optional physical parameters, combined with a linear first-order path; FM-DeepONet injects an interaction block into the branch–trunk features of DeepONet. The design is motivated by a second-order Taylor expansion of the target function/operator, with the authors explicitly stating that this is an inductive-bias argument, not a universal-approximation proof. Experiments report that FM-PINN improves relative L2 error on 16 of 18 selected multidimensional smooth PDE problems, and that FM-Operator achieves substantial error reductions on several nonlinear conservation-law and shock benchmarks while using roughly 43K parameters. The paper also reports five-seed operator results, parameter-matched ablations, a DeepONet+statistics control, and an appendix exploring sensitivity of the FM-PINN/SPINN comparison to the manufactured-solution family.","tokens_in":33506,"tokens_out":5034,"duration_ms":52060,"significance":"If the central mechanism claim holds, the paper offers a simple, parameter-efficient architectural module that could improve neural PDE solvers on problems with strong cross-field dependencies. The manuscript has real strengths: it is candid that the Taylor expansion is only a motivation, it reports multi-seed statistics for the main operator experiments, it includes several thoughtful ablations (short DeepONet, compact FM-DeepONet, DeepONet with appended statistics), and its Appendix D explicitly tests how the FM-PINN/SPINN ranking changes with solution structure. These features raise the evidentiary bar relative to many architecture-proposal papers. However, the causal attribution for FM-Operator's gains is not fully supported: no experiment removes the bi-interaction pooling while keeping the rest of the FM-Operator architecture. In addition, the FM-PINN evidence is based on single runs and on a narrow family of manufactured solutions, so the broad wording of the abstract overstates what the current experiments establish.","major_comments":[{"comment":"The central causal claim—that explicit pairwise cross-field interactions drive FM-Operator's accuracy gains—is not tested. Tables 15–17 compare FM-Operator against reduced-size DeepONet, compact FM-DeepONet, and DeepONet with appended statistics, but no ablation removes the bi-interaction pooling z_BI from FM-Operator while retaining the per-field affine embeddings, MLP decoder, and linear first-order path. A control that replaces z_BI with, e.g., elementwise sum or concatenation of the same embeddings is needed. Without it, the gains could come from the embedding-plus-MLP architecture itself rather than from the interaction products. This is load-bearing for the paper's title and for the attribution in §6 to “how the models construct cross-field interactions.”","section":"§5, “Parameter-Controlled Ablations”; Eq. (6) and Appendix A"},{"comment":"The FM-PINN comparisons are single-run. Many reported differences are small (1D Burgers 0.96%, 2D Poisson 3.64%, 4D Poisson 3.99%) and could easily be within run-to-run stochastic variation, especially for PINN training with AdamW and randomized LHS sampling. The paper itself states in §6 that the PINN comparisons are based on single runs. Given that the abstract and introduction claim that FM-PINN “improves the approximation accuracy for smooth high-order PDEs,” the central PINN claim needs either multi-seed statistics with confidence intervals or a substantially softened conclusion.","section":"§5, Table 2; §6 Limitation"},{"comment":"The broad FM-PINN claim is fragile with respect to the benchmark design. Appendix D shows that the relative ranking of FM-PINN and SPINN reverses when the manufactured solution changes from a sum of separable sines (Setting S) to a quadratic-phase solution (Setting Q), even though the PDE operator, optimizer, network scale, and evaluation grid are unchanged. The main FM-PINN evaluation uses only the separable-sine family, which is a narrow and structurally favorable test set. The manuscript's own appendix concludes that the evidence points to a “solution-structure-dependent regime rather than a universal ranking.” The abstract and Section 1 should be scoped accordingly, or additional manufactured-solution families should be included in the main evaluation.","section":"Appendix D, Table 19 and Eq. (14)"},{"comment":"The main text's “16 of 18” FM-PINN summary omits the full set of results reported in Appendix C. The appendix shows substantial negative results in 1D: convection–diffusion 1D is 48.62% worse and reaction–diffusion 1D is 295.37% worse than PINN, while Table 2 restricts attention mainly to equations with two or more spatial coordinates. This selective presentation inflates the apparent generality of the FM-PINN improvement. The main text should either report the complete set of benchmarks or explicitly state the exclusion criterion and acknowledge the 1D failures.","section":"Appendix C, Table 8 vs. main Table 2"}],"minor_comments":[{"comment":"The caption says the best mean is “shown in blue,” but the table appears in plain text. If color is used in the final PDF, ensure it is also indicated by boldface or an asterisk for accessibility.","section":"§5 / Table 3 caption"},{"comment":"Typo: “FM-PINN CLinear Advection 4D” should be “FM-PINN Linear Advection 4D.”","section":"Appendix C, Figure 13 caption"},{"comment":"The KS benchmark is described only briefly. The domain length, initial-condition normalization, and sensor/output grid should be stated precisely, since the KS results are an important exception to the FM-Operator advantage.","section":"§5, Kuramoto–Sivashinsky setup"},{"comment":"The “2D diagonal lift” benchmarks appear in the results table but are not defined in the experimental setup. Please provide the construction of these two-dimensional operator inputs.","section":"Appendix C, Table 12"},{"comment":"No code or data release is mentioned. Given the number of bespoke numerical reference solvers and benchmark configurations, a public implementation would substantially aid reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest and contains useful ablations, but the central causal claim needs one additional control experiment (FM-Operator without interaction pooling) and the PINN claims need either multi-seed evidence or careful scoping. The authors' own Appendix D is the strongest signal that the main claims are currently too broad. I would support publication after these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a legitimate extension of factorization-machine interactions to PINNs and neural operators, with operator-learning results that are solid enough to take seriously. The main caveat is that the FM-Operator comparisons don't fully isolate the bi-interaction pooling from the rest of the new architecture, and the FM-PINN half is thinner than the abstract suggests.\n\nWhat's new: the bi-interaction pooling itself is from He & Chua's NFM, but adapting it to continuous PDE coordinates and to field-partitioned operator inputs (sensors, query, statistics, parameters) is a reasonable and apparently useful move. The Taylor-expansion motivation is correct, and the paper is honest that it's an inductive-bias argument, not an approximation guarantee. The empirical work includes five-seed operator runs, parameter-matched ablations (short DeepONet, compact FM-DeepONet, DeepONet+stats), a Shift-DeepONet comparison, and a split-vs-non-split query study. The operator gains — around 38–54% error reduction on conservation laws at ~40% of DeepONet's parameters — are the strongest part, and they're reported with means and sample standard deviations.\n\nSoft spots. The stress-test concern is valid: FM-Operator is a new architecture (embeddings + pooling + MLP decoder + linear path), and no control removes just the pooling — say, replacing pairwise products with an elementwise sum of the same embeddings — while keeping the rest. FM-DeepONet partially covers this because it preserves the DeepONet inner-product structure and the interaction block still helps on four of five benchmarks; so the interaction mechanism has some support beyond FM-Operator. But for the headline FM-Operator results, the causal attribution is under-supported. One more ablation would settle it.\n\nSecond, the FM-PINN results are single-run, and the appendix contains negative 1D numbers (reaction-diffusion -295%, convection-diffusion -48.6%) that don't appear in the main table. The abstract's 'improves smooth high-order PDEs' overreaches against those numbers, though the paper does disclose the 1D degeneracy later. The table should be complete, and the claims toned down. Appendix D's finding that the SPINN comparison reverses under a different manufactured solution is a useful warning about benchmark fragility.\n\nThird, no code or data. For an empirical paper, that limits verification.\n\nBottom line: this paper deserves a serious referee. The operator half is a genuine empirical study with useful ablations; the FM-PINN half needs seeds, full results, and modest claims; and the causal role of the interaction pooling needs one clean control. I'd send it to review with major-revision expectations.","headline":"A useful, honestly reported extension of NFM interactions to PDE solving; the operator results are solid enough to take seriously, but the FM-Operator causal claim needs one more control and the FM-PINN half is thinner than the abstract suggests.","tokens_in":33965,"tokens_out":4138,"would_cite":true,"duration_ms":45231,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explicit pairwise feature interactions lift neural PDE solvers","keywords":["feature interaction","factorization machines","physics-informed neural networks","neural operators","DeepONet","bi-interaction pooling","conservation laws","sharp gradients"],"falsifier":"Run the same FM-PINN versus SPINN protocol on a family of manufactured solutions with smoothly varying CP rank and interaction strength (e.g., interpolation between Setting S and Setting Q). If, on non-axis-aligned solutions, rank-128 SPINN still beats FM-PINN in two dimensions, or if FM-Operator's advantage over DeepONet disappears when sensor statistics are appended to DeepONet under matched seeds, the central claims would be weakened. More directly: a single benchmark family with large mixed derivatives but where a plain MLP already matches FM-Operator would falsify the claim that explicit","tokens_in":33044,"feed_emoji":"🧮","tokens_out":3656,"duration_ms":35426,"temperature":0.7,"pith_summary":"This paper argues that neural networks for partial differential equations improve when they explicitly form all pairwise products between their input features — spatial coordinates, time, sensor samples, query points, and physical parameters — before deep processing. The authors build a single module, adapted from factorization machines, that embeds each field and pools the Hadamard products of all pairs. They show that the module, added to PINNs, lowers the relative L2 error in 16 of 18 tested PDEs, with reductions up to 63% on high-dimensional linear advection, and, as a standalone neural operator (FM-Operator), beats a 108K-parameter DeepONet on several shock-dominated equations while using only about 43K parameters. The aim is to show that cross-feature dependence, not raw capacity, is what these problems reward. The paper is explicit that the benefit is conditional: on Helmholtz, low-dimensional smooth problems, and Kuramoto–Sivashinsky, the interaction module gives no consistent edge.","feed_headline":"Pairwise feature products cut neural-PDE error by up to 63%","feed_subtitle":"Same block lifts PINNs on high-dimensional PDEs and beats DeepONet on shocks at 40% of parameters.","key_machinery":"The central object is the bi-interaction pooling layer, inherited from neural factorization machines. Each input field Fi (a coordinate, sensor vector, query vector, or parameter) is mapped through its own affine embedding Wi Fi + βi to a shared dimension r, and the layer returns z_BI = Σ_{i<j} e_i ⊙ e_j, which can be computed as 1/2[(Σ e_i)^⊙2 − Σ e_i^⊙2]. This layer makes all pairwise cross-field products explicit before an MLP decoder, at rank at most r per pair. The paper's Taylor-expansion analysis shows these products mirror the mixed second-derivative terms of the solution map, which explains why the module helps when mixed derivatives are large — as in high-dimensional coordinates or","core_discovery":"The central claim is that the solution maps of many PDEs contain substantial cross-field dependence — between spatial coordinates, between space and time, and between the input function and the query location — and that a network that constructs these pairwise products before nonlinear processing exploits that structure better than a plain concatenation MLP. The paper's bi-interaction pooling computes, for a set of M input fields, the vector sum of all pairwise Hadamard products of their embeddings; this is motivated by a blockwise second-order Taylor expansion, where the cross-field Hessian terms δFiᵀ Hij δFj correspond exactly to the pooled products. On smooth high-dimensional PINN problem","pith_inferences":["A testable extension: an adaptive gating mechanism that adds interaction terms only for field pairs with large estimated mixed derivatives could turn the conditional benefit from a fixed module into a learnable one, possibly recovering the negative cases (2D Darcy, reaction–diffusion, Helmholtz).","The strong performance on conservation laws suggests that higher-order (third-degree) interaction features, or interactions between summary statistics and local query neighborhoods, might further improve shock tracking; the paper's Taylor motivation caps at second order.","If the benchmark-dependence worry is taken seriously, a productive next experiment is to sample manufactured solutions with varying CP rank and measure when FM-PINN overtakes rank-128 SPINN; the paper's Appendix D provides the protocol for exactly such a sweep.","The 'no consistent advantage on smooth operator learning' caveat implies that the module's value lies in problems where sensor–query coupling is strong; this could be used as a selection criterion when deploying the method to new PDE families."],"forward_implications":["If correct, a single architectural module provides a parameter-efficient way to improve PINN accuracy on high-dimensional smooth PDEs, with the gain growing with the number of coordinate fields (from one pair in 2D to ten in 5D).","The FM-Operator result implies that explicit sensor–query interactions are a strong inductive bias for shock-dominated problems, achieving lower error than a much larger DeepONet; the improvement is not reproduced by just adding sensor statistics to DeepONet's input.","The benefit is conditional: on Helmholtz equations, low-dimensional smooth problems, and Kuramoto–Sivashinsky, the interaction module offers no consistent advantage, so the paper implies that interaction terms should be added selectively rather than universally.","The comparison with Shift-DeepONet suggests that explicit cross-field products can be more effective for moving-front tracking than input-dependent coordinate transformations of trunk basis functions, at roughly a third of the parameters.","The paper's own Appendix D cautions that architecture rankings can reverse when the manufactured solution changes structure, so the gains are solution-structure-dependent rather than universal — a direct corollary, not an external critique."],"fun_headline_variants":["Feature-interaction PINNs ace shock-dominated PDEs","Pairwise products sharpen neural PDE solvers on hard gradients","FM modules boost PINNs and operators on conservation laws, not smooth","Cross-field couplings improve physics-informed learning on stiff problems","Feature interaction layers help neural PDEs tackle discontinuities"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the chosen benchmark PDEs and manufactured solutions fairly represent the families where pairwise feature interactions matter; the paper's own Appendix D shows that switching a manufactured solution from separable sines to quadratic phases reverses the ranking between FM-PINN and SPINN, so conclusions drawn from one solution family may not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Feature-interaction PINNs ace shock-dominated PDEs","Pairwise products sharpen neural PDE solvers on hard gradients","FM modules boost PINNs and operators on conservation laws, not smooth","Cross-field couplings improve physics-informed learning on stiff problems","Feature interaction layers help neural PDEs tackle discontinuities"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000555,"raw_usage":{"total_tokens":2455,"prompt_tokens":697,"completion_tokens":1758,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":1678}},"tokens_in":441,"tokens_out":1758,"duration_ms":16560,"temperature":1.0,"reasoning_tokens":1678,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T00:26:54.931528+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same FM-PINN versus SPINN protocol on a family of manufactured solutions with smoothly varying CP rank and interaction strength (e.g., interpolation between Setting S and Setting Q). If, on non-axis-aligned solutions, rank-128 SPINN still beats FM-PINN in two dimensions, or if FM-Operator's advantage over DeepONet disappears when sensor statistics are appended to DeepONet under matched seeds, the central claims would be weakened. More directly: a single benchmark family with large mixed derivatives but where a plain MLP already matches FM-Operator would falsify the claim that explicit","supporting_citations":[],"review_version":1}