Pith. sign in

REVIEW 3 major objections 5 minor 18 cited by

Walrus claims to be the most accurate learned emulator of continuum dynamics to date, with top results on 56 of 65 metrics across 26 tasks, enabled by a jittering trick that suppresses aliasing artifacts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 21:18 UTC pith:42LIIEDI

load-bearing objection A serious and reproducible foundation-model paper whose main 3D accuracy claim rests on one-to-three test trajectories and no error bars. the 3 major comments →

arxiv 2511.15684 v2 pith:42LIIEDI submitted 2025-11-19 cs.LG cs.AIcs.CE

Walrus: A Cross-Domain Foundation Model for Continuum Dynamics

Michael McCabe , Payel Mukhopadhyay , Tanya Marwah , Bruno Regaldo-Saint Blancard , Francois Rozet , Cristiana Diaconu , Lucas Meyer , Kaze W. K. Wong
show 17 more authors
Hadi Sotoudeh Alberto Bietti Irina Espejo Rio Fear Siavash Golkar Tom Hehir Keiya Hirashima Geraud Krawezik Francois Lanusse Rudy Morel Ruben Ohana Liam Parker Mariel Pettee Jeff Shen Kyunghyun Cho Miles Cranmer Shirley Ho
This is my paper
classification cs.LG cs.AIcs.CE
keywords foundation modelcontinuum dynamicsPDE emulationpatch jitteringaliasingautoregressive stabilitymulti-physics pretraining2D and 3D simulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a single transformer, pretrained on a deliberately diverse collection of fluid-like continuum simulations, can become a general emulator: given a short history of snapshots it predicts the next state across 2D and 3D problems spanning six scientific domains. The central new mechanism is patch jittering, a random spatial shift applied before tokenization and inverted after reconstruction; harmonic analysis shows that for bandlimited periodic signals the expected output is the unaliased solution, and empirically it reduces long-horizon error in most pretraining scenarios. If the claim holds, one shared 1.3-billion-parameter model, finetuned on modest data, outperforms existing physics foundation models on most short- and long-horizon tasks, and diversity of pretraining data matters more than low pretraining loss. The result matters because it suggests the foundation-model recipe can work for physical simulation, where heterogeneous data and unstable rollouts have blocked it.

Core claim

On the paper's own terms, Walrus is the most accurate learned emulator of continuum dynamics to date: after pretraining on nineteen datasets with sixty-three state variables, finetuned Walrus achieves the lowest error on 56 of 65 tracked metrics across 26 tasks, with average one-step VRMSE reductions of roughly 64% versus prior foundation models. The authors attribute this primarily to three design choices: patch jittering, which randomizes patch-grid alignment to suppress aliasing-induced grid artifacts; treating 2D data as randomly oriented planes inside 3D tensors, so one model handles both dimensions and 3D transfer improves; and compute-adaptive tokenization with load-balanced distribut

What carries the argument

Patch jittering: a random shift of each input snapshot before patch-based tokenization, inverted after reconstruction. For bandlimited signals on periodic domains, the paper proves that the expected composed operation equals the unaliased filtered signal, because the Fourier shift factor makes the alias terms vanish by the Cauchy integral theorem. In the network this amounts to randomly re-aligning the patch grid every step at negligible cost; it is the mechanism that keeps autoregressive rollouts on the grid rather than accumulating high-frequency artifacts.

Load-bearing premise

Patch jittering is assumed to remove aliasing artifacts on real nonlinear, nonperiodic, finite-sample data even though the proof only covers bandlimited, translation-invariant signals on periodic domains.

What would settle it

Run the same jitter/no-jitter ablation on a nonperiodic, nonlinear benchmark, for example Rayleigh-Benard convection or viscoelastic instability, and measure median full-trajectory VRMSE over many rollouts: if the median improvement of roughly 54% does not reproduce and high-frequency artifact energy is not reduced, the load-bearing stability mechanism is no better than noise in that setting.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single pretrained continuum emulator can be finetuned on a new physical task with 500K samples and beat models trained with comparable or larger budgets on short and medium rollouts.
  • Patch jittering is a cheap, model-agnostic stabilizer: applying random spatial shifts during patchification and inverting them on output should reduce long-horizon error in any patch-based autoregressive emulator.
  • Pretraining diversity, including random time-stride sampling and 2D-as-3D projection, improves downstream transfer even when it worsens the pretraining loss itself.
  • Exposing the model to 3D data during pretraining is what enables competitive 3D downstream performance, since 2D-only pretraining gives only a small boost on a first 3D task.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If patch jittering's benefit really comes from ensemble-averaged unaliasing, then averaging several jittered forward passes at inference time should further suppress alias error; the paper reports only single-sample jitter.
  • The same Fourier-shift argument applies to any strided resampling operation, so jittering may also stabilize other learned PDE surrogates and generative models that use convolutional patchification.
  • A testable consequence of the 2D-as-plane augmentation is that the model should be nearly invariant to axis-aligned rotations and reflections of test inputs; a model trained with fixed orientations would lose accuracy under such transforms.
  • Because Walrus infers hidden parameters like advection speed from a short history of snapshots, its practical advantage over one-step models should grow for downstream tasks with more unobserved physics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Walrus, a 1.3B-parameter transformer-based foundation model for continuum dynamics, pretrained on nineteen datasets spanning astrophysics, geoscience, rheology, plasma physics, acoustics, and classical fluids. The key technical contributions are: (i) 'patch jittering,' a stochastic input translation claimed to reduce autoregressive rollout instability through a harmonic-analysis argument; (ii) joint 2D–3D training by embedding 2D data as randomly oriented planes with tensor-law-aware augmentations; (iii) compute-adaptive tokenization via convolutional stride modulation; and (iv) topology-aware sampling that improves distributed training throughput. The authors finetune Walrus on a range of 2D and 3D downstream tasks, report improvements over MPP-A ViT-L, Poseidon-L, and DPOT-H on most metrics, and provide ablations showing that diversity-focused pretraining helps downstream transfer.

Significance. If the empirical results hold, Walrus is a meaningful step toward general-purpose neural emulators for continuum physics: it is the first foundation model of this scale to jointly handle 2D and 3D data, and the released code and weights will be valuable for the community. The paper's strengths include a clear engineering narrative, careful ablations of the pretraining strategy, and a theoretical motivation (though not a proof) for patch jittering. However, the headline claim that Walrus is 'the most accurate foundation model for continuum simulation to date' rests on benchmark comparisons where key 3D test sets contain only one to three trajectories and where no uncertainty quantification is provided. Better statistical grounding is needed before the central claim can be considered established.

major comments (3)
  1. [§5.1, Tables 6 and 8] The 3D downstream results that drive the headline claim are based on extremely small test sets. Table 6 lists PNS with 8 total trajectories and RSG with 29. With the stated 80/10/10 split when standard splits are unavailable (Section 5), the test set is approximately 1 and 3 trajectories, respectively. Table 8 reports median VRMSE values (e.g., PNS one-step: 0.2013 vs. 0.2394) without confidence intervals, bootstrap estimates, or per-trajectory results. A single outlier trajectory can easily flip these rankings. Since the Conclusion's '56/65 tracked metrics' includes these 3D tasks, the robustness of the main empirical claim is not established for the most expensive benchmarks. Please report per-trajectory errors, confidence intervals, or at least a leave-one-out sensitivity analysis, and specify exactly how many test trajectories were used.
  2. [§3.2, Eqs. (4)–(7)] The theoretical derivation of patch jittering assumes a bandlimited, translation-invariant signal on a periodic domain and an average over a continuous distribution of shifts. The authors themselves state that the ideal setting is 'complicated by slow empirical convergence of the expectation as well as the presence of boundary conditions and nonlinearities' (§3.2). Despite this, jittering is presented as a core stabilization contribution and is credited with the 89% improvement across pretraining datasets (Table 7). The empirical evidence in Table 7 is a median over 20 trajectories with no error bars, confidence intervals, or multiple seeds. Given that the formal guarantee does not apply to the actual data, the empirical support needs to be stronger before the stability gains can be considered robust.
  3. [Conclusion] The paper's central summary claim—'top results in 56/65 tracked metrics across 26 unique continuum emulation tasks'—is not derivable from the reported tables. The set of 'tracked metrics' is never defined, and no mapping from the tables to the 56/65 count is provided. This is a load-bearing claim, as it is the main quantitative summary of the paper's contribution. Please add an explicit accounting (per dataset, per horizon, per metric) that shows how ties, non-applicable cells, and the varying numbers of horizons are counted.
minor comments (5)
  1. [Throughout] There are several typos and formatting issues, e.g., 'Hierachical' → 'Hierarchical' (Section 3.1), 'boudnaries' → 'boundaries' (Section C.2), 'Emultation' → 'Emulation' (Section 5.1), and 'Skelenton' → 'Skeleton' (dataset name, Table 6). Please proofread.
  2. [§B.3.1] The hyperparameter grid for baseline finetuning lists a duplicated learning rate: [1e-3, 5e-4, 1e-4, 5e-4, 1e-5] contains 5e-4 twice. This may be a typo.
  3. [Table 3] The caption says 'dark font signifies closer to best performance' but the table rendering does not show any dark font. Please either adjust the caption or the table formatting so the intended emphasis is visible.
  4. [§5.1 / Figure 14] The shaded bands in loss-over-time plots are described as ±1 standard deviation 'over the data set.' For the 3D tasks, where the test set has 1–3 trajectories, such shading is likely misleading. Please clarify the number of trajectories contributing to the standard deviation, and consider using pointwise per-timestep error bars instead.
  5. [§4.1 / §5.3] The paper states that 'all comparisons are trained on the combined volume Walrus was shown during both pretraining and finetuning' for in-training datasets. It would help to state explicitly how many epochs this corresponds to for each baseline, since 4.5M samples for a small dataset results in many passes and may affect the fairness of the comparison.

Circularity Check

0 steps flagged

No load-bearing circularity: the patch-jitter stabilization result is derived from first principles and tested on held-out trajectories; self-citations are background, not forcing premises.

full rationale

The paper's central novel technical claim, patch jittering, is derived in Section 3.2 from the Fourier shift theorem and Cauchy integral theorem (Eqs. 4-7), yielding an un-aliased expectation, and is then validated empirically on held-out validation trajectories (Table 7, 'improvement on 17/19 pretraining datasets'). The empirical gain is not obtained by fitting any constant from the theory, so there is no fitted-input-called-prediction or self-definitional reduction. The downstream comparisons (Section 5.1) and pretraining-strategy ablations (Section 5.3) are empirical evaluations on fixed benchmark metrics, not derivations from the paper's own assumptions. Several components are inherited from prior work with overlapping authorship (MPP: McCabe et al. 2023a; aliasing/stability heuristics: McCabe et al. 2023b; adaptive patching: Mukhopadhyay et al. 2025; The Well: Ohana et al. 2025), but these are transparently cited engineering/background choices and are not used as unverified premises that force the paper's conclusions. The paper itself flags the main limitation of the patch-jitter theory: 'In realistic settings, this theoretical observation is complicated by slow empirical convergence of the expectation as well as the presence of boundary conditions and nonlinearities' (Section 3.2). That is an honest scope limitation, not circularity. The skeptic's concern about the 'most accurate' claim resting on very small 3D test sets (PNS has 8 trajectories, RSG 29) and no confidence intervals is a legitimate empirical-verification weakness, but it concerns statistical robustness of benchmark comparisons, not circular derivation. Overall, no prediction in the paper reduces by construction to a fitted input or to a self-citation chain.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced. The central result is empirical; the only theoretical component is the patch-jittering analysis, whose assumptions are explicit. Free parameters listed are design choices of the training procedure, not fitted physical constants.

free parameters (3)
  • jitter shift sampling range = 0 to pad_total (pixels)
    The shift is drawn uniformly over a range tied to the effective kernel/stride; chosen by design, not fitted to data, but the range affects de-aliasing effectiveness.
  • time-stride sampling range = 1–5 simulation steps
    Used during pretraining to force the model to infer time scale; range chosen by hand.
  • 2D/3D token budget = 32² for 2D, 16³ for 3D
    Adaptive-compute tokenization targets these token counts to balance datasets; chosen to fit hardware.
axioms (4)
  • domain assumption Bandlimited, periodic signal assumption in patch-jittering derivation
    Section 3.2 assumes U ∈ L²(T) is bandlimited (Û[k]=0 for |k|>N/2) and uses continuous translation s∼T to derive Eq. 7. This idealization is acknowledged as not holding exactly for real data.
  • domain assumption Boundary conditions can be summarized by open/closed/periodic topology
    Section C.2 encodes boundaries as binary mask channels only for open/closed/periodic; the model cannot learn detailed physical boundary conditions.
  • domain assumption Dataset splits are representative
    Where no canonical splits exist, the paper uses 80/10/10 splits by trajectory (Section 5), assuming the test split is not biased; for PNS (8 trajectories) and RSG (29) these splits yield very few test trajectories.
  • domain assumption Euclidean tensor-law augmentation validity
    Augmentations of rotations/reflections are applied only to Euclidean domains (Section 4.1). For spherical/log-spherical data (PlanetSWE, PNS, RSG) the model must generalize without such augmentations; this is untested.

pith-pipeline@v1.3.0-alltime-deepseek · 34130 in / 12707 out tokens · 119148 ms · 2026-08-03T21:18:05.180997+00:00 · methodology

0 comments
read the original abstract

Foundation models have transformed machine learning for language and vision, but achieving comparable impact in physical simulation remains a challenge. Data heterogeneity and unstable long-term dynamics inhibit learning from sufficiently diverse dynamics, while varying resolutions and dimensionalities challenge efficient training on modern hardware. Through empirical and theoretical analysis, we incorporate new approaches to mitigate these obstacles, including a harmonic-analysis-based stabilization method, load-balanced distributed 2D and 3D training strategies, and compute-adaptive tokenization. Using these tools, we develop Walrus, a transformer-based foundation model developed primarily for fluid-like continuum dynamics. Walrus is pretrained on nineteen diverse scenarios spanning astrophysics, geoscience, rheology, plasma physics, acoustics, and classical fluids. Experiments show that Walrus outperforms prior foundation models on both short and long term prediction horizons on downstream tasks and across the breadth of pretraining data, while ablation studies confirm the value of our contributions to forecast stability, training throughput, and transfer performance over conventional approaches. Code and weights are released for community use.

Figures

Figures reproduced from arXiv: 2511.15684 by Alberto Bietti, Bruno Regaldo-Saint Blancard, Cristiana Diaconu, Francois Lanusse, Francois Rozet, Geraud Krawezik, Hadi Sotoudeh, Irina Espejo, Jeff Shen, Kaze W. K. Wong, Keiya Hirashima, Kyunghyun Cho, Liam Parker, Lucas Meyer, Mariel Pettee, Michael McCabe, Miles Cranmer, Payel Mukhopadhyay, Rio Fear, Ruben Ohana, Rudy Morel, Shirley Ho, Siavash Golkar, Tanya Marwah, Tom Hehir.

Figure 1
Figure 1. Figure 1: Walrus is a modern transformer incorporating novel stabilizing techniques and recent adaptive-compute methods to learn from a highly diverse set of physical dynamics. Walrus takes as input a short sequence of snapshots and predicts the next step in the sequence. Shared Encoder-Decoder. All physical systems S of a given dimensionality d share a single encoder and decoder block forcing the model to learn gen… view at source ↗
Figure 2
Figure 2. Figure 2: Patch jittering (middle) re￾duces the accumulation of high fre￾quency artifacts (top) allowing for more stable long-term forecasts. The strided convolution with filter g can be exactly written in the frequency domain as: yˆ[k] = ˆg[k]ˆu[k] + P X−1 j=1 gˆ[k + jM]ˆu[k + jM] (2) recalling that due to periodicity uˆ[k] = ˆu[k + jN] and yˆ[k] = ˆy[k+jM] ∀j ∈ Z, this amounts to the summation of the aliased frequ… view at source ↗
Figure 3
Figure 3. Figure 3: All raw data is projected into 3D by ap [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Tokenization and distribution strate￾gies are carefully balanced to ensure each copy of Walrus receives similarly sized buckets of to￾kens within each synchronous block, minimizing deadweight and maximizing throughput. However, these steps alone are insufficient to reduce all discrepancies in workload. En￾coder/decoder costs still vary by input size, though this can be partially accounted for by simply rep… view at source ↗
Figure 5
Figure 5. Figure 5: Visualizing the prediction of a finetuned [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Loss (median VRMSE) on 2D downstream tasks after finetuning each foundation model. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: VRMSE on downstream 3D tasks. Lower is bet [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: One-step VRMSE in limited data scenarios denoted by restricted numbers of samples. (Top) [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Pretraining VRMSE from HalfWalrus and naively trained models. From pretraining alone, it appears that the less diverse “naive” strategy outperforms HalfWalrus, but this trend is reversed on down￾stream tasks in [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Stages of the Walrus model. Walrus uses recently developed adaptive-compute methods to balance cost between datasets on top of a modern transformer backbone and novel stabilization approaches. A MODEL DETAILS Walrus can be broken into the following stages shown in order in [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Stages of the Walrus model. Walrus uses recently developed adaptive-compute methods to balance cost between datasets on top of a modern transformer backbone and novel stabilization approaches. Algorithm 1 Patch Jittering with Adaptive-Compute Patching Require: Data u, p1, p2 encoder convolutional kernel sizes, s1, s2 encoder convolutional kernel strides, b boundary identifiers. 1: function PADANDJITTER(u,… view at source ↗
Figure 12
Figure 12. Figure 12: Stages of the Walrus model. Walrus uses recently developed adaptive-compute methods to balance cost between datasets on top of a modern transformer backbone and novel stabilization approaches. To reconstruct the fields in grid space, the reverse procedure is followed: transposed convolutions using the same p, s values perform upsampling, the jitter is inverted, and padded cells are cropped. The only compl… view at source ↗
Figure 13
Figure 13. Figure 13: Comparsion between Walrus and the Poseidon-L model on linear advection of smooth initial conditions. Without history, the task becomes degenerate as it is impossible to infer the missing system parameters. One important point when comparing emulation models is the mode of operation. In practice, there is a wide spectrum of information one can assume the model has access to. For instance, given full observ… view at source ↗
Figure 14
Figure 14. Figure 14: VRMSE over time for downstream tasks. Shaded region denotes [PITH_FULL_IMAGE:figures/full_fig_p032_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: VRMSE over time for tasks in training distribution. Shaded region denotes [PITH_FULL_IMAGE:figures/full_fig_p034_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Rayleigh-Benard Convection in 2D. GT t = 0 Pred t = 15 t = 30 t = 45 t = 60 Shear Flow tracer [PITH_FULL_IMAGE:figures/full_fig_p035_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Shear flow in 2D. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Active matter dynamics in 2D. GT t = 0 Pred MultiQuadrantsP (APE) t = 15 t = 30 t = 45 t = 60 density [PITH_FULL_IMAGE:figures/full_fig_p036_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Euler multi-quadrants problem with periodic boundaries in 2D solved with absolute [PITH_FULL_IMAGE:figures/full_fig_p036_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Euler multi-quadrants problem with periodic boundaries in 2D solved without APE. [PITH_FULL_IMAGE:figures/full_fig_p036_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Euler multi-quadrants problem with open boundaries in 2D solved with APE. [PITH_FULL_IMAGE:figures/full_fig_p037_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Euler multi-quadrants problem with periodic boundaries in 2D solved without APE. [PITH_FULL_IMAGE:figures/full_fig_p037_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Acoustic scattering through a maze in 2D. [PITH_FULL_IMAGE:figures/full_fig_p037_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Acoustic scattering through medium with sharply varying inclusions in 2D. [PITH_FULL_IMAGE:figures/full_fig_p038_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Acoustic scattering through medium with discontinuous density in 2D. [PITH_FULL_IMAGE:figures/full_fig_p038_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Flow around object with boundaries defined by spherical harmonics. [PITH_FULL_IMAGE:figures/full_fig_p038_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Gray-Scott diffusion reaction in 2D. GT Pred GT t = 0 Pred t = 8 t = 16 t = 24 t = 32 Staircase pressure_re pressure_im [PITH_FULL_IMAGE:figures/full_fig_p039_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: ”Helmholtz Staircase” of wave propagation over infinite periodic surface in 2D. [PITH_FULL_IMAGE:figures/full_fig_p039_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Turbulent radiative layer in 2D. GT Pred GT t = 0 Pred t = 15 t = 30 t = 45 t = 60 PlanetSWE height velocity_theta [PITH_FULL_IMAGE:figures/full_fig_p040_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Shallow water equations over approximate earth topography (PlanetSWE). [PITH_FULL_IMAGE:figures/full_fig_p040_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Flow around complex obstacle defined by skelenton in 2D. [PITH_FULL_IMAGE:figures/full_fig_p040_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: Compressible Euler equations initialized at Richtmyer-Meshkov instability in 2D. From [PITH_FULL_IMAGE:figures/full_fig_p041_32.png] view at source ↗
Figure 33
Figure 33. Figure 33: Instability in viscoelastic flow in 2D. GT t = 0 Pred t = 9 t = 19 t = 28 t = 38 ConditionedINS tracer [PITH_FULL_IMAGE:figures/full_fig_p041_33.png] view at source ↗
Figure 34
Figure 34. Figure 34: Particles embedded in buoyant incompressible flow in 2D. From PDEArena. [PITH_FULL_IMAGE:figures/full_fig_p041_34.png] view at source ↗
Figure 35
Figure 35. Figure 35: Multi-phase flow in which varying liquids are heated from below to form bubbles in 2D. [PITH_FULL_IMAGE:figures/full_fig_p042_35.png] view at source ↗
Figure 36
Figure 36. Figure 36: Magnetohydrodynamic turbulence in 3D. GT t = 0 Pred t = 15 t = 30 t = 45 t = 60 Postmerger Neutron Star electron fraction [PITH_FULL_IMAGE:figures/full_fig_p042_36.png] view at source ↗
Figure 37
Figure 37. Figure 37: Aftermath of neutron star merger. Visualized slice in 2D, but simulated in 3D. [PITH_FULL_IMAGE:figures/full_fig_p042_37.png] view at source ↗
Figure 38
Figure 38. Figure 38: Compressible Navier-Stokes in 3D with “random” initial conditions. From PDEBench. [PITH_FULL_IMAGE:figures/full_fig_p043_38.png] view at source ↗
Figure 39
Figure 39. Figure 39: Compressible Navier-Stokes in 3D with “turbulent” initial conditions. From PDEBench. [PITH_FULL_IMAGE:figures/full_fig_p043_39.png] view at source ↗
Figure 40
Figure 40. Figure 40: Cooling of the turbulent interstellar medium in 3D. [PITH_FULL_IMAGE:figures/full_fig_p043_40.png] view at source ↗
Figure 41
Figure 41. Figure 41: Rayleigh-Taylor Instability in 3D. GT t = 0 Predicted t = 11 t = 23 t = 35 t = 47 Supernova (3D) - velocity [PITH_FULL_IMAGE:figures/full_fig_p044_41.png] view at source ↗
Figure 42
Figure 42. Figure 42: Supernova explosion in 3D. GT t = 0 Predicted t = 15 t = 30 t = 45 t = 60 T R L (3 D) - lo g( d d x ) [PITH_FULL_IMAGE:figures/full_fig_p044_42.png] view at source ↗
Figure 43
Figure 43. Figure 43: Turbulent radiative layer in 3D. 44 [PITH_FULL_IMAGE:figures/full_fig_p044_43.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Function graph transformers universally approximate operators between function spaces

    cs.LG 2026-05 unverdicted novelty 8.0

    Function graph transformers use graph measures to provide a measure-theoretic framework where standard transformer components universally approximate operators between function spaces while preserving single-valued fu...

  2. Breakeven complexity: A new perspective on neural partial differential equation solvers

    cs.LG 2026-05 unverdicted novelty 7.0

    Breakeven complexity is introduced to evaluate neural PDE solvers by total end-to-end cost, with results indicating they become advantageous for harder problems such as higher dimensions, longer rollouts, and higher R...

  3. Discovering Physical Directions in Weight Space: Composing Neural PDE Experts

    cs.LG 2026-05 unverdicted novelty 7.0

    Fine-tuning neural PDE operators to regime endpoints reveals a physical direction in weight space that CCM uses to compose accurate merged models for new or extrapolated regimes from metadata or short prefixes.

  4. Flowers: A Warp Drive for Neural PDE Solvers

    cs.LG 2026-02 conditional novelty 7.0

    Flowers, a neural PDE solver built from multihead warps alone, beats same-size Fourier/convolution/attention baselines on most tested flow and wave benchmarks.

  5. OmniMol: Transferring Particle Physics Knowledge to Molecular Dynamics with Point-Edge Transformers

    physics.chem-ph 2026-01 unverdicted novelty 7.0

    OmniMol transfers a billion-jet pre-trained PET foundation model from HEP to molecular dynamics via an interaction-matrix attention bias, delivering strong performance on the oMol dataset with minimal fine-tuning and ...

  6. NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamics

    cs.CV 2026-06 unverdicted novelty 6.0

    NEXUS introduces a graph-based neural energy-field model that derives forces from scalar energy and dissipation terms to achieve physically consistent contact-rich 3D dynamics.

  7. Small Models, Strong Priors: Architectural Inductive Bias for Parameter-Efficient Neural PDE Solvers

    cs.LG 2026-05 unverdicted novelty 6.0

    WaveLiT combines wavelet tokenization, linear attention, and multiscale pyramids to produce parameter-efficient neural PDE solvers that match much larger models on TheWell benchmarks.

  8. AOT-POT: Adaptive Operator Transformation for Large-Scale PDE Pre-training

    cs.LG 2026-05 unverdicted novelty 6.0

    AOT-POT adaptively reshapes complex PDE solution operators via input-dependent transformations and parallel stream mixing to enable effective large-scale pre-training, yielding SOTA results on 12 benchmarks with minim...

  9. Autoregressive One-Step Generative Modeling for Dynamical System Forecasting

    cs.LG 2026-05 unverdicted novelty 6.0

    MeLISA delivers one-step blockwise generative forecasting for dynamical systems that improves short-term accuracy and long-horizon statistical fidelity over neural operators while matching or exceeding their inference speed.

  10. Autoregressive One-Step Generative Modeling for Dynamical System Forecasting

    cs.LG 2026-05 conditional novelty 6.0

    MeLISA extends pixel-space MeanFlow to one-step window-conditioned autoregressive forecasting, improving long-horizon turbulence statistics over neural-operator baselines.

  11. Transformers for dynamical systems learn transfer operators in-context

    cs.LG 2026-02 unverdicted novelty 6.0

    Small transformers learn to forecast unseen dynamical systems in-context by using delay embeddings to recover the manifold and forecasting its invariant sets via a transfer-operator strategy.

  12. Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs

    cs.LG 2026-02 conditional novelty 6.0

    A geometry-conditioned Whitney-form neural network that solves a learned discrete conservation law improves out-of-distribution geometry generalization for steady-state PDEs compared with regression-based neural operators.

  13. Otter Weather: Skillful and Computationally Efficient Medium-Range Weather Forecasting

    cs.LG 2026-06 unverdicted novelty 5.0

    Otter Weather is a spatiotemporal model that outperforms NWP baselines by 9.6% at 24h lead with under 3.5 A100-days training and extends efficiency gains to probabilistic forecasting via CRPS.

  14. Sparse probes and murky physics: a case study of interpretability challenges in a foundation model for continuum dynamics

    cs.LG 2026-06 unverdicted novelty 5.0

    Case study applies SAE probing with enstrophy triage to a continuum-dynamics foundation model and reports intermittent feature consistency that does not align with standard physics while linking some output discrepanc...

  15. jNO: A JAX Library for Neural Operator and Foundation Model Training

    cs.LG 2026-05 unverdicted novelty 4.0

    jNO introduces a unified JAX tracing system for data-driven and physics-informed neural operator training that compiles domains, residuals, losses, and diagnostics into one pipeline.

  16. Replay-Based Continual Learning for Physics-Informed Neural Operators

    cs.LG 2026-05 unverdicted novelty 4.0

    A replay-based continual learning strategy for physics-informed neural operators mitigates catastrophic forgetting on prior physical problems while enabling efficient adaptation to new data using only physical constraints.

  17. Foundation Models for Astrophysics

    astro-ph.IM 2026-08 conditional novelty 3.0

    Astronomical 'foundation models' largely reuse transformers and self-supervised pretraining, but evidence of transfer to new instruments, populations, or tasks remains rare; the paper argues such evidence, not archite...

  18. Towards a Foundation Model for the Martian Atmosphere

    astro-ph.EP 2026-05 unverdicted novelty 3.0

    The paper reviews data sources, physical models, downstream applications, and AI techniques to outline considerations for building a foundation model for the Martian atmosphere.

Reference graph

Works this paper leans on

15 extracted references · 1 linked inside Pith · cited by 17 Pith papers

  1. [1]

    This was an artifact of merging several implementations rather than an intentional decision

    Encoder and processor using different activations - The encoder layers all use GELU activations (Hendrycks & Gimpel, 2016) while the processor layers use SwiGLU. This was an artifact of merging several implementations rather than an intentional decision. On small scale experiments, we saw little difference from correcting this, so it was left as is

    2016

  2. [2]

    However, in multiple dimensions, the padding will overlap in corners resulting in additive corner handling

    Additive corner handling - In the boundary padding examples, we discussed the one- dimensional case where boundary topology is denoted by appended one-hot encoded chan- nels. However, in multiple dimensions, the padding will overlap in corners resulting in additive corner handling. Changing this to a cross-product of boundary tokens was not explored. B TR...

    2014

  3. [3]

    For non-periodic boundaries, padding is implemented with additional channels containing binary masks for each topological boundary type

    Periodic - There is not truly a boundary here and the neighboring points are on the opposite side of the domain. For non-periodic boundaries, padding is implemented with additional channels containing binary masks for each topological boundary type. Open and closed boudnaries are processed as separate channels so that the model can learn to treat each dif...

    2021

  4. [4]

    The model is forced to learn to infer the time step from history

    Variable time-stride: We sample trajectories spaced by varying ∆t where this ranges from 1 to 5 simulation steps. The model is forced to learn to infer the time step from history. This information is not provided explicitly

  5. [5]

    time-conditioning

    Tensor-aware rotations: For all data defined on Euclidean domains, we perform additional augmentation using rotations sampled from the axis-aligned subset of SO(3). As mentioned in the main text, all geometric transformations are performed both on the spatial layout and on tensor-valued fields to ensure physical consistency. For non-Walrus models in exper...

    2001

  6. [6]

    Pretraining data (Table 5) - Data used during pretrainingWalrus

  7. [7]

    C.1 DATA TRANSFORMATIONS The following elementwise transformations were applied to input fields independent of normalization

    Downstream tasks (Table 6) - Data used for finetuning task which had never been seen by Walrus during initial pretraining, though Conditioned Incompressible NS from PDEArena was used during DPOT training and CE-RM from PDEGym was part of the same data collection as Poseidon’s pretraining. C.1 DATA TRANSFORMATIONS The following elementwise transformations ...

  8. [9]

    •turbulence gravity cooling

    temperature: log10 - reduce long tail of strictly positive field in accordance with domain conventions. •turbulence gravity cooling

  9. [11]

    •turbulent radiative layer 3D

    temperature: log10 - reduce long tail of strictly positive field in accordance with domain conventions. •turbulent radiative layer 3D

  10. [12]

    density: log10 - reduce long tail of strictly positive field in accordance with domain conventions

  11. [13]

    C.2 BOUNDARYHANDLING In the case of periodic boundary conditions, the computations described in Section 3.2 can be performed exactly

    temperature: log10 - reduce long tail of strictly positive field in accordance with domain conventions. C.2 BOUNDARYHANDLING In the case of periodic boundary conditions, the computations described in Section 3.2 can be performed exactly. As Walrus is intended to be general-purpose and not constrained to settings where the boundary conditions are exactly k...

  12. [14]

    We do not know what is beyond the boundary

    Open - The domain extends beyond the sub-domain we are currently viewing. We do not know what is beyond the boundary

  13. [15]

    The model is unaware of what conditions are enforced at this barrier

    Closed - There is some form of barrier marking this as the limit of the domain. The model is unaware of what conditions are enforced at this barrier. 29

  14. [17]

    random” initial conditions. From PDEBench. GT t = 0 Predicted t = 2 t = 4 t = 6 t = 9 CNS Turb (3D) - density Figure 39: Compressible Navier-Stokes in 3D with “turbulent

    though the impact of this is not explored in this work. D MODES OFOPERATION FORDYNAMICSMODELS Ground Truth Walrus Poseidon Walrus Error Poseidon Error 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 Figure 13: Comparsion between Walrus and the Poseidon-L model on linear advection of smooth initial conditions. Without history, the task becomes degenerate as it...

    arXiv 2024

  15. [2024]

    PERIODIC

    URLhttps://arxiv.org/abs/2210.07182. Ronak Tali, Ali Rabeh, Cheng-Hau Yang, Mehdi Shadkhah, Samundra Karki, Abhisek Upadhyaya, Suriya Dhakshinamoorthy, Marjan Saadati, Soumik Sarkar, Adarsh Krishnamurthy, Chinmay Hegde, Aditya Balu, and Baskar Ganapathysubramanian. Flowbench: A large scale benchmark for flow simulation over complex geometries, 2024. URL h...

    Pith/arXiv arXiv 2024