REVIEW 4 major objections 5 minor 25 references
DPNO: A Dual Path Architecture For Neural Operator
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper argues that arranging neural operator blocks along two parallel paths—a residual path and a dense path—consistently improves approximation, cutting relative $L^2$ error by 30–40 percent on standard PDE benchmarks.
desk verdict Plausible transfer of the DPN dual-path idea to FNO and DeepONet with consistent reported gains, but capacity is not controlled, so the architectural claim is conditional. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dual path composition rule, adapted from dual path networks. With $\mathcal{G}_k$ denoting any basic operator block, the ResNet-like path evolves $u_{k+1}(x)=\mathcal{G}_k(u_k)(x)+u_k(x)$, while the DenseNet-like path evolves $v_{k+1}(x)=\mathcal{G}_k([v_0,\dots,v_k])(x)$, and the outputs of both paths are fused by a neural network to produce the final solution. The residual path preserves and refines earlier features, and the dense path reuses every intermediate output so new representations can be explored without discarding old ones. In DeepONet this rule is applied to the trunk net; in FNO the basic block is one Fourier integral operator followed by a nonlinearity. The rule is block-agnostic, which is what lets the paper attribute the gains to the connection pattern rather than to any particular operator block.
What would settle it
Re-run the six comparisons with baselines and dual path models matched for parameter count and FLOPs, for example by shrinking the dual path's width or depth until its parameter count equals the baseline's; if the relative $L^2$ advantage vanishes or reverses at matched capacity, the connection pattern is not the cause.
Extended reading notes
Core claim
The authors' central claim is that a dual path organization of operator blocks—one path that adds each block's output to its input, one path that concatenates each block's output with all previous outputs—produces a stronger solution approximator than the same blocks arranged in the usual feedforward stack. On the six model-benchmark pairs tested, the dual path version of DeepONet and FNO achieves a lower relative $L^2$ error in every case; the largest improvements are 39.58% for DeepONet on Burgers, 36.73% for FNO on Darcy flow, and 33.77% for FNO on Burgers. For DeepONet, the dual path is applied only to the trunk network, motivated by reading trunk outputs as basis functions whose number the dense path effectively increases. For FNO, one Fourier integral block plus activation is the basic unit rearranged along the two paths. The paper presents the architecture as an empirical, transferable design pattern and does not claim a proof of why it works.
Load-bearing premise
The argument assumes the dual path models and their single-path baselines have comparable capacity, so that the error drops are caused by the connection pattern rather than by extra parameters or compute; the paper reports no parameter counts or FLOPs, and the described block counts (three dense-path plus four residual-path Fourier blocks versus four plain Fourier blocks) leave that comparability open.
Editorial extensions
If this is right
- Users of DeepONet or FNO can try to improve accuracy without changing the operator block: rewiring the blocks along the two paths is the whole intervention.
- The largest reported gains, 39.58% on Burgers for DeepONet and 36.73% on Darcy for FNO, make dual path stacking a default candidate when single-path stacks hit their accuracy ceiling.
- Because the design treats the operator block as an interchangeable unit, it extends to newer stackable operators, including transformer-based neural operators, which the paper names as future work.
- For DeepONet the dense path enlarges the trunk's effective set of basis functions, so the architecture can be read as a parameter-aware way to enrich the approximation space.
Reading between the lines
- The natural next experiment, not run in the paper, is a capacity-matched comparison: shrink the dual path model's width or depth until its parameter count equals the baseline's, then re-measure the six relative $L^2$ drops.
- Because the dense path concatenates every previous block output, its memory grows with depth; bottleneck or channel-reduction variants of the same dual path idea may keep most of the accuracy gain at lower cost.
- If the effect is genuinely about feature reuse, the same dual path rule should improve other operator blocks beyond DeepONet and FNO; testing one block per family is a minimal demonstration rather than evidence of universality.
- The paper's basis-function reading of the DeepONet trunk suggests a concrete probe: compare the effective span of the dual path trunk's outputs with that of a plain trunk to see whether the dense path is really adding independent basis directions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a dual path architecture (DPNO) for neural operators, organizing basic operator blocks along a ResNet-like residual path and a DenseNet-like dense path. The construction is instantiated for two representative operator paradigms: DeepONet and FNO. The method is evaluated on three PDE benchmarks (Burgers, Darcy flow, and 2D Navier-Stokes) and Table 1 reports relative L2 improvements over the corresponding base models on all six model-benchmark pairs, e.g., 39.58% for DeepONet on Burgers and 36.73% for FNO on Darcy. The authors conclude that the dual path connection pattern is a versatile and effective design for neural operators, while explicitly acknowledging in Section 5 that the method lacks rigorous theoretical proof.
Significance. If the reported improvements survive capacity-controlled comparisons, the paper would make a useful empirical contribution by identifying connection topology as a design axis that is orthogonal to the choice of operator block, with evidence from both the encoder-decoder (DeepONet) and kernel-integral (FNO) paradigms. The paper also has the merit of being transparent about the absence of theoretical justification. However, the current evidence is weakened by the lack of parameter or FLOP counts, the absence of repeated runs and error bars, and the lack of code release; as a result, the central architectural claim is not yet cleanly isolated from model-capacity effects.
major comments (4)
- [§4, Table 1] The comparison does not control model capacity. For FNO, the baseline stacks four Fourier blocks, while the dual-path FNO uses three DenseNet-path blocks plus four ResNet-path blocks; DenseNet concatenation increases the channel dimension feeding each subsequent block, and the paper itself states that 'the deepening of DenseNet path will significantly increases the number of parameters.' For DeepONet, the dual-path trunk is built from multiple blocks while the baseline trunk is a single 4-layer, width-128 MLP, and if the trunk output dimension grows through concatenation, the branch output dimension must also grow to keep the dot product defined. Since no parameter counts or FLOPs are reported, the 14%–40% relative L2 improvements in Table 1 could be explained by increased capacity rather than by the dual-path topology. The authors should add parameter-matched baselines (e.g., wider or deeper single-path FNO and DeepONet with comparable parameter counts) and report parameter counts and FLOPs for all models.
- [§3.4] The DeepONet construction is underspecified in a way that affects the capacity comparison. The text says 'we only apply dual path architecture on trunk net' and claims to 'decrease the number of model simultaneously,' but the DenseNet path concatenates trunk block outputs, which increases the trunk output dimension. Because the DeepONet prediction is the dot product of branch and trunk outputs, the branch net must be modified to produce a matching output dimension. The paper does not describe how the branch is adjusted, how its parameter count changes, or whether the total parameter count is indeed decreased. This needs to be clarified and, ideally, quantified.
- [§4, Table 1 and Figs. 2–6] All experimental results are single runs with no error bars, no seeds, and no repeated trials. Since the reported improvements range from 7.38% to 39.58%, it is impossible to assess whether the differences are statistically meaningful. The authors should report mean and standard deviation over multiple seeds, state how the test checkpoint is selected (e.g., best validation loss vs. final epoch), and provide the training configuration (loss function, normalization, data splits) needed for reproducibility.
- [§2.3 and §5] The relationship to the densely connected integral autoencoder of reference [8] is not sufficiently clarified. The paper states that this earlier work already uses a densely connected multi-block structure inspired by DenseNet and that the present work is 'enlightened by their work,' yet no experiment compares DPNO against that prior multi-block operator. Without such a comparison, or at least a careful discussion of what the addition of the parallel ResNet path contributes beyond [8], the novelty and advantage of the proposed design over existing multi-block operators is not established.
minor comments (5)
- [Throughout] The manuscript contains numerous typos and language errors, e.g., 'staking' for 'stacking,' 'probelm' for 'problem,' 'start-of-art' for 'state-of-the-art,' and 'For three dimensions problem' when describing the 2D Navier-Stokes experiment. These should be corrected in a thorough revision.
- [§3.2] The architecture equations are presented without equation numbers and are later referred to as 'the above equation' or 'the equation is given by,' which makes it hard to refer to specific formulas. Numbering the equations would improve clarity.
- [References] In the version of the manuscript I reviewed, the reference list appears badly corrupted (many entries are unreadable fragments). The authors must ensure that the final version contains a clean, correctly formatted reference list.
- [§4] The text says 'For DeepONet, we use a 4 layers block with width 128 in the middle and GeLU activation to construct both branch net and trunk net,' but it does not specify the branch and trunk input/output dimensions or the final projection layer, which are needed to reproduce the experiments. Adding a short architecture table or explicit dimension flow would help.
- [§4, Figs. 2, 3, 5] The training/testing loss curves are described in the captions as showing training and testing processes, but the legends and axis labels are not explained in the text. The reader cannot tell which curves correspond to which model or whether the reported numbers come from the best or final epoch.
Circularity Check
No circularity found: the paper reports an empirical architecture comparison with no derivation that reduces to its own inputs.
full rationale
The paper makes no claim that could reduce to its own input. It proposes a dual-path arrangement of neural operator blocks, defines the ResNet-like and DenseNet-like update equations, and then compares the resulting empirical test errors against DeepONet and FNO baselines on standard Burgers, Darcy, and Navier-Stokes benchmarks. The baselines are standard external models, the proposed models are trained on the same datasets, and the reported relative L2 improvements are direct experimental measurements rather than quantities forced by construction. No parameter is fitted to a subset of data and then reported as a prediction of a closely related quantity. The authors' citation of Dual Path Networks [10] is prior external work, not a self-citation chain, and the analogy to DPN is used as architectural inspiration rather than as a uniqueness theorem or as a substitute for the experiments. The paper's own acknowledgments that the effectiveness 'still lacks rigorous theoretical proof' and that the DenseNet path 'significantly increases the number of parameters' are honest limitations, but they concern capacity control and theoretical justification, not circularity. The absence of parameter counts or FLOPs is a soundness or correctness concern about whether the gains come from capacity rather than architecture; it is not a circular-reasoning defect. The central claim is an empirical benchmark result that stands or falls on the experiments, so the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (5)
- DenseNet path depth =
3 operator blocks
- ResNet path depth =
4 operator blocks
- DeepONet block configuration =
4 layers, width 128
- Baseline FNO depth =
4 Fourier blocks
- Training hyperparameters =
AdamW, lr=0.001, 1000 epochs
assumptions (4)
- standard math Universal approximation theorem for nonlinear operators (Chen and Chen 1995)
- domain assumption Fourier integral operator parameterization as given by Li et al. [7]
- domain assumption Dataset generation protocol from [7]
- ad hoc to paper Transfer of ResNet and DenseNet benefits from image classification to operator blocks
Cite this review
Pith. "Pith review of DPNO: A Dual Path Architecture For Neural Operator." pith.science (2026). https://pith.science/paper/UE5X6MII
@misc{pith2026250712719,
author = {Pith},
title = {Pith review of: DPNO: A Dual Path Architecture For Neural Operator},
year = {2026},
howpublished = {\url{https://pith.science/paper/UE5X6MII}},
note = {Machine review of arXiv:2507.12719}
}
read the original abstract
Neural operators have emerged as a powerful tool for solving partial differential equations (PDEs) and other complex scientific computing tasks. However, the performance of single operator block is often limited, thus often requiring composition of basic operator blocks to achieve better per-formance. The traditional way of composition is staking those blocks like feedforward neural networks, which may not be very economic considering parameter-efficiency tradeoff. In this pa-per, we propose a novel dual path architecture that significantly enhances the capabilities of basic neural operators. The basic operator block is organized in parallel two paths which are similar with ResNet and DenseNet. By introducing this parallel processing mechanism, our architecture shows a more powerful feature extraction and solution approximation ability compared with the original model. We demonstrate the effectiveness of our approach through extensive numerical experi-ments on a variety of PDE problems, including the Burgers' equation, Darcy Flow Equation and the 2d Navier-Stokes equation. The experimental results indicate that on certain standard test cas-es, our model achieves a relative improvement of over 30% compared to the basic model. We also apply this structure on two standard neural operators (DeepONet and FNO) selected from different paradigms, which suggests that the proposed architecture has excellent versatility and offering a promising direction for neural operator structure design.
Reference graph
Works this paper leans on
-
[8]
In g nc d n w k f disc iz i n-invariant ning
Y ng Zh ng ng, Z w i Sh n, nd H izh Y ng. “In g nc d n w k f disc iz i n-invariant ning”. In: J n f M chin ning R s ch 23.286 (2022), . 1–45
work page 2022
-
[1]
Uns vis d d ning algorithm for PDE -based forward and inverse prob- ms
L h B nd i S ch n. “Uns vis d d ning algorithm for PDE -based forward and inverse prob- ms”. In: Xiv preprint arXiv:1904.05417 (2019)
arXiv 2019
-
[2]
M d d c i n nd n n w ks f m ic PDEs
K shik Bh ch . “M d d c i n nd n n w ks f m ic PDEs”. In: Th SMAI j n of computational mathematics 7 (2021), pp. 121–157
work page 2021
-
[3]
Th d Ri z m h d: d ning b s d n m ic g i hm f s ving v i i n b- ms
Bing Y . “Th d Ri z m h d: d ning b s d n m ic g i hm f s ving v i i n b- ms”. In: C mm nic i ns in M h m ics nd S is ics 6.1 (2018), . 1–12
work page 2018
-
[4]
M zi R issi, P is P dik is, nd G g E K ni d kis. “Ph sics-informed neural networks: A deep learning f m w k f s ving f w d nd inv s b ms inv ving n n in i diff n i q i ns”. In: Journal of Computational physics 378 (2019), pp. 686–707
work page 2019
-
[5]
: G h k n n w k f i diff n i q i ns
Z ng i i . “ : G h k n n w k f i diff n i q i ns”. In: Xiv in arXiv:2003.03485 (2020)
arXiv 2020
-
[6]
ning n n in s vi D b s d n h niv s xim i n h m f s
“ ning n n in s vi D b s d n h niv s xim i n h m f s”. In: m chin in ig nc 3.3 (2021), . 218–229
work page 2021
-
[7]
F i n f m ic i diff n i q i ns
Z ng i i . “F i n f m ic i diff n i q i ns”. In: Xiv in arXiv:2010.08895 (2020)
arXiv 2020
Show all 25 references
-
[9]
T ns v : A f s nsf m s v f d s n g n g m i s
H ix W . “T ns v : A f s nsf m s v f d s n g n g m i s”. In: Xiv in arXiv:2402.02366 (2024)
2024 arXiv
-
[10]
D h n w ks
Y n ng Ch n . “D h n w ks”. In: Adv nc s in n inf m i n c ssing s s ms 30 (2017)
2017
-
[11]
s f cc ing sci n ific sim i ns nd d sign
K m Azizz d n sh i . “ s f cc ing sci n ific sim i ns nd d sign”. In: Reviews Physics 6.5 (2024), pp. 320–328
2024
-
[12]
Univ s xim i n n n in s b n n w ks wi h - bi c iv i n f nc i ns nd i s ic i n d n mic s s ms
Ti n ing Ch n nd H ng Ch n. “Univ s xim i n n n in s b n n w ks wi h - bi c iv i n f nc i ns nd i s ic i n d n mic s s ms”. In: IEEE ns c i ns n n n w ks 6.4 (1995), pp. 911–917
1995
-
[13]
V i b -in d n w ks
Mich P s h f , Tim D R ck, nd Siddh h Mish . “V i b -in d n w ks”. In: Xiv preprint arXiv:2205.11404 (2022)
2022 arXiv
-
[14]
B sis n w k: A n n w k -based model for learning nonlinear s vi n b sis
ing H nd W n i n . “B sis n w k: A n n w k -based model for learning nonlinear s vi n b sis”. In: w ks 164 (2023), . 21–37
2023
-
[15]
W v n f s ving m ic i diff n i q i ns in c m i n m ch nics b ms
T s T i nd S vik Ch k b . “W v n f s ving m ic i diff n i q i ns in c m i n m ch nics b ms”. In: C m M h ds in A i d M ch nics nd Engin - ing 404 (2023), p. 115783
2023
-
[16]
U -net: Convolutional networks for biomedical image s gm n i n
f R nn b g , Phi i Fisch , nd Th m s B x.“U -net: Convolutional networks for biomedical image s gm n i n”. In: M dic im g c m ing nd c m -assisted intervention–MICCAI 2015: 18th interna- tional conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer....
2015
-
[17]
D sid ning f im g c gni i n
K iming H . “D sid ning f im g c gni i n”. In: P c dings f h IEEE c nf nc n computer vision and pattern recognition. 2016, pp. 770–778
2016
-
[18]
D ns c nn c d c nv i n n w ks
G H ng . “D ns c nn c d c nv i n n w ks”. In: P c dings f h IEEE c nf nc n computer vision and pattern recognition. 2017, pp. 4700–4708
2017
-
[19]
An im g is w h 16x16 w ds: T nsf m s f im g c gni i n sc
A x D s vi ski . “An im g is w h 16x16 w ds: T nsf m s f im g c gni i n sc ”. In: arXiv preprint arXiv:2010.11929 (2020)
2020 arXiv
-
[20]
U -no: U-sh d n s
Md Ashiq R hm n, Z ch E R ss, nd K m Azizz d n sh i. “U -no: U-sh d n s”. In: arXiv preprint arXiv:2204.11127 (2022)
2022 arXiv
-
[21]
Ch s nsf m : F i g kin
Sh h C . “Ch s nsf m : F i g kin”. In: Adv nc s in n inf m i n c ssing s s- tems 34 (2021), pp. 24924–24940
2021
-
[22]
Gn : A g n n nsf m f ning
Zh ngk i H . “Gn : A g n n nsf m f ning”. In: In n i n C n- ference on Machine Learning. PMLR. 2023, pp. 12556–12569
2023
-
[23]
Im v d ning b h g n n i n
Zi ng Xi . “Im v d ning b h g n n i n”. In: Xiv in Xiv:2310.12487 (2023)
2023 arXiv
-
[24]
ning: A g i hms nd n sis
ik B K v chki, S m n h , nd And w M S . “ ning: A g i hms nd n sis”.In: arXiv preprint arXiv:2402.15715 (2024)
2024 arXiv
-
[25]
L.C. Evans. Partial Differential Equations. Graduate studies in mathematics. American Mathematical Society,
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.