Pith. sign in

REVIEW 4 major objections 5 minor 25 references

DPNO: A Dual Path Architecture For Neural Operator

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read This paper argues that arranging neural operator blocks along two parallel paths—a residual path and a dense path—consistently improves approximation, cutting relative $L^2$ error by 30–40 percent on standard PDE benchmarks.

desk verdict Plausible transfer of the DPN dual-path idea to FNO and DeepONet with consistent reported gains, but capacity is not controlled, so the architectural claim is conditional. read the letter →

arxiv 2507.12719 v1 pith:UE5X6MII submitted 2025-07-17 math.NA cs.NA

classification math.NAcs.NA MSC 65M9968T07
keywords neuraloperatordualpatharchitectureDeepONetFourierpartialdifferentialequationsResNetDenselearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the way basic neural operator blocks are connected is itself a design choice: by running the same kind of block along a ResNet-like residual path and a DenseNet-like dense path in parallel, the resulting dual path model approximates PDE solution operators more accurately than the usual single-path stack. The claim is tested on two operator families, DeepONet and Fourier Neural Operator (FNO), across Burgers, Darcy flow, and 2D Navier-Stokes, and the dual path versions win on all six model-benchmark pairs, with relative $L^2$ error reductions of 39.58% (DeepONet on Burgers), 36.73% (FNO on Darcy), and 33.77% (FNO on Burgers). If true, this gives a simple, model-agnostic recipe: improve an existing neural operator by reorganizing its blocks rather than redesigning the operator block. It also suggests that feature reuse across depths, previously developed for image classifiers, transfers to function-space learning.

What carries the argument

The load-bearing object is the dual path composition rule, adapted from dual path networks. With $\mathcal{G}_k$ denoting any basic operator block, the ResNet-like path evolves $u_{k+1}(x)=\mathcal{G}_k(u_k)(x)+u_k(x)$, while the DenseNet-like path evolves $v_{k+1}(x)=\mathcal{G}_k([v_0,\dots,v_k])(x)$, and the outputs of both paths are fused by a neural network to produce the final solution. The residual path preserves and refines earlier features, and the dense path reuses every intermediate output so new representations can be explored without discarding old ones. In DeepONet this rule is applied to the trunk net; in FNO the basic block is one Fourier integral operator followed by a nonlinearity. The rule is block-agnostic, which is what lets the paper attribute the gains to the connection pattern rather than to any particular operator block.

What would settle it

Re-run the six comparisons with baselines and dual path models matched for parameter count and FLOPs, for example by shrinking the dual path's width or depth until its parameter count equals the baseline's; if the relative $L^2$ advantage vanishes or reverses at matched capacity, the connection pattern is not the cause.

Watch

Extended reading notes

Core claim

The authors' central claim is that a dual path organization of operator blocks—one path that adds each block's output to its input, one path that concatenates each block's output with all previous outputs—produces a stronger solution approximator than the same blocks arranged in the usual feedforward stack. On the six model-benchmark pairs tested, the dual path version of DeepONet and FNO achieves a lower relative $L^2$ error in every case; the largest improvements are 39.58% for DeepONet on Burgers, 36.73% for FNO on Darcy flow, and 33.77% for FNO on Burgers. For DeepONet, the dual path is applied only to the trunk network, motivated by reading trunk outputs as basis functions whose number the dense path effectively increases. For FNO, one Fourier integral block plus activation is the basic unit rearranged along the two paths. The paper presents the architecture as an empirical, transferable design pattern and does not claim a proof of why it works.

Load-bearing premise

The argument assumes the dual path models and their single-path baselines have comparable capacity, so that the error drops are caused by the connection pattern rather than by extra parameters or compute; the paper reports no parameter counts or FLOPs, and the described block counts (three dense-path plus four residual-path Fourier blocks versus four plain Fourier blocks) leave that comparability open.

Editorial extensions

If this is right

  • Users of DeepONet or FNO can try to improve accuracy without changing the operator block: rewiring the blocks along the two paths is the whole intervention.
  • The largest reported gains, 39.58% on Burgers for DeepONet and 36.73% on Darcy for FNO, make dual path stacking a default candidate when single-path stacks hit their accuracy ceiling.
  • Because the design treats the operator block as an interchangeable unit, it extends to newer stackable operators, including transformer-based neural operators, which the paper names as future work.
  • For DeepONet the dense path enlarges the trunk's effective set of basis functions, so the architecture can be read as a parameter-aware way to enrich the approximation space.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The natural next experiment, not run in the paper, is a capacity-matched comparison: shrink the dual path model's width or depth until its parameter count equals the baseline's, then re-measure the six relative $L^2$ drops.
  • Because the dense path concatenates every previous block output, its memory grows with depth; bottleneck or channel-reduction variants of the same dual path idea may keep most of the accuracy gain at lower cost.
  • If the effect is genuinely about feature reuse, the same dual path rule should improve other operator blocks beyond DeepONet and FNO; testing one block per family is a minimal demonstration rather than evidence of universality.
  • The paper's basis-function reading of the DeepONet trunk suggests a concrete probe: compare the effective span of the dual path trunk's outputs with that of a plain trunk to see whether the dense path is really adding independent basis directions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a dual path architecture (DPNO) for neural operators, organizing basic operator blocks along a ResNet-like residual path and a DenseNet-like dense path. The construction is instantiated for two representative operator paradigms: DeepONet and FNO. The method is evaluated on three PDE benchmarks (Burgers, Darcy flow, and 2D Navier-Stokes) and Table 1 reports relative L2 improvements over the corresponding base models on all six model-benchmark pairs, e.g., 39.58% for DeepONet on Burgers and 36.73% for FNO on Darcy. The authors conclude that the dual path connection pattern is a versatile and effective design for neural operators, while explicitly acknowledging in Section 5 that the method lacks rigorous theoretical proof.

Significance. If the reported improvements survive capacity-controlled comparisons, the paper would make a useful empirical contribution by identifying connection topology as a design axis that is orthogonal to the choice of operator block, with evidence from both the encoder-decoder (DeepONet) and kernel-integral (FNO) paradigms. The paper also has the merit of being transparent about the absence of theoretical justification. However, the current evidence is weakened by the lack of parameter or FLOP counts, the absence of repeated runs and error bars, and the lack of code release; as a result, the central architectural claim is not yet cleanly isolated from model-capacity effects.

major comments (4)
  1. [§4, Table 1] The comparison does not control model capacity. For FNO, the baseline stacks four Fourier blocks, while the dual-path FNO uses three DenseNet-path blocks plus four ResNet-path blocks; DenseNet concatenation increases the channel dimension feeding each subsequent block, and the paper itself states that 'the deepening of DenseNet path will significantly increases the number of parameters.' For DeepONet, the dual-path trunk is built from multiple blocks while the baseline trunk is a single 4-layer, width-128 MLP, and if the trunk output dimension grows through concatenation, the branch output dimension must also grow to keep the dot product defined. Since no parameter counts or FLOPs are reported, the 14%–40% relative L2 improvements in Table 1 could be explained by increased capacity rather than by the dual-path topology. The authors should add parameter-matched baselines (e.g., wider or deeper single-path FNO and DeepONet with comparable parameter counts) and report parameter counts and FLOPs for all models.
  2. [§3.4] The DeepONet construction is underspecified in a way that affects the capacity comparison. The text says 'we only apply dual path architecture on trunk net' and claims to 'decrease the number of model simultaneously,' but the DenseNet path concatenates trunk block outputs, which increases the trunk output dimension. Because the DeepONet prediction is the dot product of branch and trunk outputs, the branch net must be modified to produce a matching output dimension. The paper does not describe how the branch is adjusted, how its parameter count changes, or whether the total parameter count is indeed decreased. This needs to be clarified and, ideally, quantified.
  3. [§4, Table 1 and Figs. 2–6] All experimental results are single runs with no error bars, no seeds, and no repeated trials. Since the reported improvements range from 7.38% to 39.58%, it is impossible to assess whether the differences are statistically meaningful. The authors should report mean and standard deviation over multiple seeds, state how the test checkpoint is selected (e.g., best validation loss vs. final epoch), and provide the training configuration (loss function, normalization, data splits) needed for reproducibility.
  4. [§2.3 and §5] The relationship to the densely connected integral autoencoder of reference [8] is not sufficiently clarified. The paper states that this earlier work already uses a densely connected multi-block structure inspired by DenseNet and that the present work is 'enlightened by their work,' yet no experiment compares DPNO against that prior multi-block operator. Without such a comparison, or at least a careful discussion of what the addition of the parallel ResNet path contributes beyond [8], the novelty and advantage of the proposed design over existing multi-block operators is not established.
minor comments (5)
  1. [Throughout] The manuscript contains numerous typos and language errors, e.g., 'staking' for 'stacking,' 'probelm' for 'problem,' 'start-of-art' for 'state-of-the-art,' and 'For three dimensions problem' when describing the 2D Navier-Stokes experiment. These should be corrected in a thorough revision.
  2. [§3.2] The architecture equations are presented without equation numbers and are later referred to as 'the above equation' or 'the equation is given by,' which makes it hard to refer to specific formulas. Numbering the equations would improve clarity.
  3. [References] In the version of the manuscript I reviewed, the reference list appears badly corrupted (many entries are unreadable fragments). The authors must ensure that the final version contains a clean, correctly formatted reference list.
  4. [§4] The text says 'For DeepONet, we use a 4 layers block with width 128 in the middle and GeLU activation to construct both branch net and trunk net,' but it does not specify the branch and trunk input/output dimensions or the final projection layer, which are needed to reproduce the experiments. Adding a short architecture table or explicit dimension flow would help.
  5. [§4, Figs. 2, 3, 5] The training/testing loss curves are described in the captions as showing training and testing processes, but the legends and axis labels are not explained in the text. The reader cannot tell which curves correspond to which model or whether the reported numbers come from the best or final epoch.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper reports an empirical architecture comparison with no derivation that reduces to its own inputs.

full rationale

The paper makes no claim that could reduce to its own input. It proposes a dual-path arrangement of neural operator blocks, defines the ResNet-like and DenseNet-like update equations, and then compares the resulting empirical test errors against DeepONet and FNO baselines on standard Burgers, Darcy, and Navier-Stokes benchmarks. The baselines are standard external models, the proposed models are trained on the same datasets, and the reported relative L2 improvements are direct experimental measurements rather than quantities forced by construction. No parameter is fitted to a subset of data and then reported as a prediction of a closely related quantity. The authors' citation of Dual Path Networks [10] is prior external work, not a self-citation chain, and the analogy to DPN is used as architectural inspiration rather than as a uniqueness theorem or as a substitute for the experiments. The paper's own acknowledgments that the effectiveness 'still lacks rigorous theoretical proof' and that the DenseNet path 'significantly increases the number of parameters' are honest limitations, but they concern capacity control and theoretical justification, not circularity. The absence of parameter counts or FLOPs is a soundness or correctness concern about whether the gains come from capacity rather than architecture; it is not a circular-reasoning defect. The central claim is an empirical benchmark result that stands or falls on the experiments, so the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical, so the ledger contains no fitted physical constants. The main costs are architectural hyperparameters chosen by hand, reliance on prior operator frameworks, external dataset generation from [7], and an assumed transfer of ResNet/DenseNet benefits from image classification to operator learning. No new physical entities are introduced.

free parameters (5)
  • DenseNet path depth = 3 operator blocks
    Hand-chosen due to GPU memory constraints; no ablation or sensitivity study is reported.
  • ResNet path depth = 4 operator blocks
    Hand-chosen alongside the DenseNet depth; this drives the capacity difference with the baseline stacks.
  • DeepONet block configuration = 4 layers, width 128
    Used for both branch and trunk nets with GeLU activation; chosen without reported tuning.
  • Baseline FNO depth = 4 Fourier blocks
    Baseline stack from [7]; the dual path version uses a different number of blocks, confounding capacity.
  • Training hyperparameters = AdamW, lr=0.001, 1000 epochs
    Fixed across models but not tuned; batch size, weight decay, and random seeds are not reported.
assumptions (4)
  • standard math Universal approximation theorem for nonlinear operators (Chen and Chen 1995)
    Underlies the DeepONet branch/trunk construction in Section 3.4; accepted as proven background.
  • domain assumption Fourier integral operator parameterization as given by Li et al. [7]
    The FNO basic block is taken as a black box from prior work in Sections 2.3 and 3.5; its approximation properties are not re-derived.
  • domain assumption Dataset generation protocol from [7]
    Inputs are sampled from Gaussian random fields and PDEs are solved numerically as in [7]; no dataset files or generation code are shipped, so the paper relies entirely on that external protocol.
  • ad hoc to paper Transfer of ResNet and DenseNet benefits from image classification to operator blocks
    The core motivation in Sections 2.2 and 3.2 assumes that architectural benefits demonstrated in computer vision transfer to operator learning, but no controlled experiment isolates this transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DPNO: A Dual Path Architecture For Neural Operator." pith.science (2026). https://pith.science/paper/UE5X6MII

@misc{pith2026250712719,
  author       = {Pith},
  title        = {Pith review of: DPNO: A Dual Path Architecture For Neural Operator},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UE5X6MII}},
  note         = {Machine review of arXiv:2507.12719}
}
read the original abstract

Neural operators have emerged as a powerful tool for solving partial differential equations (PDEs) and other complex scientific computing tasks. However, the performance of single operator block is often limited, thus often requiring composition of basic operator blocks to achieve better per-formance. The traditional way of composition is staking those blocks like feedforward neural networks, which may not be very economic considering parameter-efficiency tradeoff. In this pa-per, we propose a novel dual path architecture that significantly enhances the capabilities of basic neural operators. The basic operator block is organized in parallel two paths which are similar with ResNet and DenseNet. By introducing this parallel processing mechanism, our architecture shows a more powerful feature extraction and solution approximation ability compared with the original model. We demonstrate the effectiveness of our approach through extensive numerical experi-ments on a variety of PDE problems, including the Burgers' equation, Darcy Flow Equation and the 2d Navier-Stokes equation. The experimental results indicate that on certain standard test cas-es, our model achieves a relative improvement of over 30% compared to the basic model. We also apply this structure on two standard neural operators (DeepONet and FNO) selected from different paradigms, which suggests that the proposed architecture has excellent versatility and offering a promising direction for neural operator structure design.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 16 canonical work pages

  1. [8]

    In g nc d n w k f disc iz i n-invariant ning

    Y ng Zh ng ng, Z w i Sh n, nd H izh Y ng. “In g nc d n w k f disc iz i n-invariant ning”. In: J n f M chin ning R s ch 23.286 (2022), . 1–45

  2. [1]

    Uns vis d d ning algorithm for PDE -based forward and inverse prob- ms

    L h B nd i S ch n. “Uns vis d d ning algorithm for PDE -based forward and inverse prob- ms”. In: Xiv preprint arXiv:1904.05417 (2019)

  3. [2]

    M d d c i n nd n n w ks f m ic PDEs

    K shik Bh ch . “M d d c i n nd n n w ks f m ic PDEs”. In: Th SMAI j n of computational mathematics 7 (2021), pp. 121–157

  4. [3]

    Th d Ri z m h d: d ning b s d n m ic g i hm f s ving v i i n b- ms

    Bing Y . “Th d Ri z m h d: d ning b s d n m ic g i hm f s ving v i i n b- ms”. In: C mm nic i ns in M h m ics nd S is ics 6.1 (2018), . 1–12

  5. [4]

    Ph sics-informed neural networks: A deep learning f m w k f s ving f w d nd inv s b ms inv ving n n in i diff n i q i ns

    M zi R issi, P is P dik is, nd G g E K ni d kis. “Ph sics-informed neural networks: A deep learning f m w k f s ving f w d nd inv s b ms inv ving n n in i diff n i q i ns”. In: Journal of Computational physics 378 (2019), pp. 686–707

  6. [5]

    : G h k n n w k f i diff n i q i ns

    Z ng i i . “ : G h k n n w k f i diff n i q i ns”. In: Xiv in arXiv:2003.03485 (2020)

  7. [6]

    ning n n in s vi D b s d n h niv s xim i n h m f s

    “ ning n n in s vi D b s d n h niv s xim i n h m f s”. In: m chin in ig nc 3.3 (2021), . 218–229

  8. [7]

    F i n f m ic i diff n i q i ns

    Z ng i i . “F i n f m ic i diff n i q i ns”. In: Xiv in arXiv:2010.08895 (2020)

Show all 25 references
  1. [9]

    T ns v : A f s nsf m s v f d s n g n g m i s

    H ix W . “T ns v : A f s nsf m s v f d s n g n g m i s”. In: Xiv in arXiv:2402.02366 (2024)

  2. [10]

    D h n w ks

    Y n ng Ch n . “D h n w ks”. In: Adv nc s in n inf m i n c ssing s s ms 30 (2017)

  3. [11]

    s f cc ing sci n ific sim i ns nd d sign

    K m Azizz d n sh i . “ s f cc ing sci n ific sim i ns nd d sign”. In: Reviews Physics 6.5 (2024), pp. 320–328

  4. [12]

    Univ s xim i n n n in s b n n w ks wi h - bi c iv i n f nc i ns nd i s ic i n d n mic s s ms

    Ti n ing Ch n nd H ng Ch n. “Univ s xim i n n n in s b n n w ks wi h - bi c iv i n f nc i ns nd i s ic i n d n mic s s ms”. In: IEEE ns c i ns n n n w ks 6.4 (1995), pp. 911–917

  5. [13]

    V i b -in d n w ks

    Mich P s h f , Tim D R ck, nd Siddh h Mish . “V i b -in d n w ks”. In: Xiv preprint arXiv:2205.11404 (2022)

  6. [14]

    B sis n w k: A n n w k -based model for learning nonlinear s vi n b sis

    ing H nd W n i n . “B sis n w k: A n n w k -based model for learning nonlinear s vi n b sis”. In: w ks 164 (2023), . 21–37

  7. [15]

    W v n f s ving m ic i diff n i q i ns in c m i n m ch nics b ms

    T s T i nd S vik Ch k b . “W v n f s ving m ic i diff n i q i ns in c m i n m ch nics b ms”. In: C m M h ds in A i d M ch nics nd Engin - ing 404 (2023), p. 115783

  8. [16]

    U -net: Convolutional networks for biomedical image s gm n i n

    f R nn b g , Phi i Fisch , nd Th m s B x.“U -net: Convolutional networks for biomedical image s gm n i n”. In: M dic im g c m ing nd c m -assisted intervention–MICCAI 2015: 18th interna- tional conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer....

  9. [17]

    D sid ning f im g c gni i n

    K iming H . “D sid ning f im g c gni i n”. In: P c dings f h IEEE c nf nc n computer vision and pattern recognition. 2016, pp. 770–778

  10. [18]

    D ns c nn c d c nv i n n w ks

    G H ng . “D ns c nn c d c nv i n n w ks”. In: P c dings f h IEEE c nf nc n computer vision and pattern recognition. 2017, pp. 4700–4708

  11. [19]

    An im g is w h 16x16 w ds: T nsf m s f im g c gni i n sc

    A x D s vi ski . “An im g is w h 16x16 w ds: T nsf m s f im g c gni i n sc ”. In: arXiv preprint arXiv:2010.11929 (2020)

  12. [20]

    U -no: U-sh d n s

    Md Ashiq R hm n, Z ch E R ss, nd K m Azizz d n sh i. “U -no: U-sh d n s”. In: arXiv preprint arXiv:2204.11127 (2022)

  13. [21]

    Ch s nsf m : F i g kin

    Sh h C . “Ch s nsf m : F i g kin”. In: Adv nc s in n inf m i n c ssing s s- tems 34 (2021), pp. 24924–24940

  14. [22]

    Gn : A g n n nsf m f ning

    Zh ngk i H . “Gn : A g n n nsf m f ning”. In: In n i n C n- ference on Machine Learning. PMLR. 2023, pp. 12556–12569

  15. [23]

    Im v d ning b h g n n i n

    Zi ng Xi . “Im v d ning b h g n n i n”. In: Xiv in Xiv:2310.12487 (2023)

  16. [24]

    ning: A g i hms nd n sis

    ik B K v chki, S m n h , nd And w M S . “ ning: A g i hms nd n sis”.In: arXiv preprint arXiv:2402.15715 (2024)

  17. [25]

    L.C. Evans. Partial Differential Equations. Graduate studies in mathematics. American Mathematical Society,

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.