{"id":"18ffc0a0-0c49-4220-a3b2-932284f83dae","arxiv_id":"2607.24513","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Patch-wise function projection plus factorized space/state attention yields a Transformer PDE solver that reports state-of-the-art accuracy from 2D dynamics to million-scale 3D CFD.","lead":"Physics Transformer turns irregular physical fields into compact tokens by projecting each local patch onto learned basis functions, then mixes those tokens with factorized attention. If the gains hold, it is a practical recipe for Transformer PDE surrogates on industrial meshes without full-point attention.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The industrial-scale SOTA claims (Tables 3–4) rest partly on Transolver-3/AB-UPT numbers adopted from their papers, while DrivAerML uses the authors' own ad hoc 400/50/50 split — same-split, same-metric comparability of the decisive leaderboard is unverified.","rationale":"The reader's weakest_assumption — that a shared basis generator keeps token index m semantically aligned across patches (§3.2–3.3), justifying the factorized attention — is a legitimate conceptual soft spot, and I agree it is the weakest link in the theory narrative. But I do not think it is the most load-bearing concern for the empirical claim. If indices were badly misaligned, the same-m cross-patch attention stage would be attending mismatched states and accuracy should visibly degrade; instead the ablations (patch-count scaling in Table 6, basis-count scaling in Table 7, attention maps in Fig. 4) and the reproduced-baseline comparisons show the factorization working, and the second within-patch attention stage plus Trans-Hilbert re-patchification across layers provide partial correction mechanisms. That premise remains unproven but is empirically hedged.\n\nThe concern I elevate was mentioned by the reader only as an auditability footnote (\"some industrial baselines are partly literature-adopted\"). On inspection it is more outcome-determinative: it is unchecked (unlike the alignment premise, which the experiments indirectly test), it attaches to the paper's single most important claim (industrial-scale SOTA), and the decision margins (a C_l R² gap of 0.002, p_s gaps under 10%, no seed variance reported) are small enough that split or metric inconsistency could reverse the ranking. This is a correctness-risk concern about the evidence, not about the architecture.\n\nThe reader's verdict of CONDITIONAL with medium correctness risk already anticipates unresolved reproducibility issues, and my concern does not warrant REJECT — the method is coherent, the reproduced-baseline results in Table 2 and the efficiency/decoding results in Appendix A/B.3 stand independently, and the fix is a concrete rerunning exercise rather than a redesign. So the verdict stays CONDITIONAL, but the conditions should explicitly include same-split reproduction (or release of splits) for the adopted Transolver-3/AB-UPT numbers and multi-seed uncertainty on Tables 3–4, alongside the reader's code-release and naming-fix conditions.","tokens_in":23349,"tokens_out":3846,"duration_ms":133328,"concrete_test":"Release the exact 400/50/50 DrivAerML and NASA-CRM split files, then rerun Transolver-3 (a working training/eval pipeline already exists per Tables 11–12) on those identical splits using the paper's own relative-L2 implementation, matched 100K-point training protocol, and ≥3 seeds; recompute the Table 3 rows and Table 4 R² values. If Transolver-3's same-split errors shift toward the adopted literature values by more than the current margin (e.g., p_s falls near/below 3.35, or C_l R² rises above 0.993), the industrial SOTA claim must be qualified to a tie; if reproduced values match the adopted ones within seed noise, the concern is discharged.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is \"state-of-the-art predictive performance... through industrial-scale 3D CFD,\" and its evidence is Tables 3–4. But §B.2 states: \"We adopt the results on NASA-CRM and DrivAerML from the paper of Transolver-3 and AB-UPT,\" while §B admits \"the original dataset does not prescribe a unique learning partition for this task, we split the 500 vehicle configurations into 400 training, 50 validation, and 50 test cases.\" If Transolver-3 and AB-UPT used different train/test partitions of those 500 vehicles (there is no official split to anchor to), the adopted numbers were measured on different test sets and are not commensurable with Physics Transformer's. The authors themselves demonstrate awareness that metric mismatches matter — they re-ran GeoTransolver because its paper reported relative L1 — yet no analogous check is documented for the adopted Transolver-3/AB-UPT numbers (metric variant, subsampling of 100K points at train time, epoch budget).\n\nThis is load-bearing because the margins are small enough to flip. DrivAerML p_s: 3.35 vs 3.71 (Transolver-3); NASA-CRM C_f: 3.31 vs 3.71. Table 4 is tighter still: C_d R² ties GeoTransolver at 0.996, and C_l is 0.993 vs 0.991 — a difference far below typical seed-to-seed variance, and no multi-seed statistics are reported anywhere in the paper. A split or metric inconsistency shifting a baseline by even a few percent could reorder the industrial leaderboard, which is the paper's headline differentiator over the Transolver family. The 2D/REALM results (Table 2) appear to use reproduced baselines and are not touched by this concern, nor is the architectural contribution itself or the cached-token decoding result (Table 9, internally consistent). But \"SOTA on industrial-scale 3D CFD\" is doing real work in the abstract and §5.2, and that sub-claim currently stands on cross-paper numbers the authors did not audit.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The manuscript proposes Physics Transformer, a Transformer-based neural operator for PDE field prediction on regular grids, irregular meshes, and point clouds. Points are serialized by a Hilbert space-filling curve and split into P locality-preserving patches; within each patch, a shared, input-adaptive generator produces M softmax-normalized local basis functions (tokenization and de-tokenization bases), onto which the local field is projected to yield M \"physics tokens\" per patch. Global interaction is factorized: attention across patches within each token index m, then attention across the M tokens within each patch. A Galerkin/Petrov–Galerkin interpretation is given (§4), including a rank argument that patch-local bases provide up to PM effective basis functions versus M for global slices. Experiments cover six benchmarks (TRL2D, Gray–Scott, REALM rocket combustor, F1-Airfoil, NASA-CRM, DrivAerML) with one-step and 20-step rollout errors, integrated Cd/Cl R² on DrivAerML, ablations on ordering/patch count/basis count/mesh scale, efficiency comparisons against Transolver-3, and a cached-token arbitrary-query mechanism that decodes full-resolution meshes from a 100K-point support set with no accuracy loss (Table 9).","tokens_in":23883,"tokens_out":4961,"duration_ms":172966,"significance":"If the results hold, this is a solid contribution to Transformer-based operator learning: it identifies a real limitation of global-slice tokenization (over-smoothing of local fine-scale structure) and offers a clean fix with a plausible Galerkin-style rationale. The strongest evidence is concrete and falsifiable: six benchmarks with reproduced baselines under matched protocols, ablations over patch count P, basis count M, ordering scheme, and mesh scale; cached-token full-mesh decoding with no measurable accuracy loss (Table 9); and reported 15.3×/7.5× full-resolution inference speedups over Transolver-3 on ~8M/~15M-cell meshes (Table 12), which addresses a genuine industrial pain point (accurate Cd/Cl quadrature requires full-surface evaluation). Training configurations are tabulated (Table 10) and official implementations are used for baselines, which supports reproducibility. The arbitrary-query mechanism (§3.5, Algorithm 1) is a practically useful addition beyond the accuracy story.","major_comments":[{"comment":"§B.2 and Tables 3–4: the Transolver-3 and AB-UPT rows on NASA-CRM and DrivAerML are adopted from those papers, while the authors use their own ad hoc 400/50/50 split of DrivAerML (§B) because the dataset has no official partition. If the source papers used a different partition, metric variant, or training budget (per §B.2, Transolver-3 is trained 800/600 epochs vs. 500 here), those rows are not commensurable with the rest of the table. The authors already re-ran GeoTransolver precisely because its paper reported relative L1, so the same scrutiny should be applied to the adopted numbers. One mitigation: the reproduced, same-protocol GeoTransolver is stronger than the adopted Transolver-3 row on DrivAerML and NASA-CRM, so the headline ordering likely survives even if the adopted rows are dropped — but this should be made explicit. Requested: mark adopted rows in Tables 3–4, state the sour","section":"§B.2 / Tables 3–4"},{"comment":"No multi-seed statistics are reported anywhere in the paper. This is load-bearing specifically for Table 4, where the headline integrated-coefficient result is C_d R² = 0.996 (a tie with GeoTransolver) and C_l = 0.993 vs. 0.991 — differences well below typical seed-to-seed variance for models of this size. The pointwise field-error margins over the strongest reproduced baseline (e.g., DrivAerML p_s 3.35 vs. 3.64) are comfortable enough that the main SOTA claim is probably robust, but the Table 4 wording ('achieves R² scores above 0.99') and the boldface ordering should be supported by at least 3-seed mean/std on the DrivAerML runs, or the claim should be softened to a statistical tie.","section":"Tables 3–4, §5.2"},{"comment":"The central architectural premise — that a shared score projection W_s makes token index m semantically aligned across heterogeneous patches, so factorized attention (spatial attention per fixed m, then token attention per patch) is an adequate substitute for coupling all PM tokens — is asserted in §3.2–3.3 and underpins both the complexity story (§3.6) and the PM-basis-capacity interpretation (§4.3), but is never directly ablated. The provided ablations vary P, M, ordering, and mesh scale, but none compares against (i) full PM×PM attention at small scale, (ii) a variant without the cross-patch same-m stage, or (iii) a shuffled-index control that would test whether the same-m alignment matters at all. A controlled comparison on a small benchmark (e.g., TRL2D at P=512, M=32, where full PM attention is affordable) would either substantiate the factorization or bound what it gives up; eithe","section":"§3.2–3.3, §4.3"}],"minor_comments":[{"comment":"§3.2: the tokenization equation reads T_p = K^⊤_p V_p, but K_p is never defined; §4.2 writes the same step correctly as Φ^⊤_p V_p. Please fix the symbol in §3.2 (and check the similar 'sprojection W_s' typo in the following paragraph).","section":"§3.2"},{"comment":"§5 (first paragraph) and §5.1 ('Large-scale industrial CFD'): the model is referred to as 'SGNO' twice — an apparent leftover from another manuscript. Also 'DriverML' (§1, §B.2) and 'DrivAerML' are used inconsistently; 'papar' and 'Expect for Transolver3' in §B.3/§B.2; 'Trl2D' in Fig. 5 caption; 'traning'/'predication' in Fig. 3 caption; 'Deeper red color express' in Fig. 4 caption.","section":"§5, captions, §B"},{"comment":"Table 2 caption does not state which metric is reported per benchmark (VRMSE for TRL2D/GSRD, normalized MSE for REALM per §B.1), nor whether values are percentages; Table 3 values are stated to be percentages only in §B.1. Please annotate the tables.","section":"Tables 2–3"},{"comment":"Table 2: on Gray–Scott rollout, FFNO (0.47) beats Physics Transformer (0.49), and Table 5 shows BallTree's τ error (5.25) is slightly worse than Hilbert's (5.24) while the text claims BallTree gives a 'marginal improvement over Hilbert.' The abstract and §5.2 'state-of-the-art' phrasing should be qualified to 'best overall' rather than uniformly best.","section":"Tables 2, 5; abstract"},{"comment":"§3.6: the dominant attention term O(MP²·d_t) is quadratic in P, and the default P=1024 makes this non-trivial; Transolver-style global slices scale linearly in N·M. The efficiency tables (11–12) address wall-clock cost, but a brief FLOPs or scaling discussion clarifying when the P² term bites would make the complexity claim more precise.","section":"§3.6"},{"comment":"§3.5/Algorithm 1: for arbitrary queries, the de-tokenization weights q_q depend on optional query features a_q — please state what a_q is at previously unseen locations (geometry only? boundary conditions?) and what happens when it is unavailable, since the full-mesh DrivAerML decoding in Table 9 relies on this path.","section":"§3.5, Appendix A"},{"comment":"§5.6: 'increasing the inference scale from 100K to 1M improves the relative L2 error by less than 1%' is ambiguous (absolute percentage points or relative improvement?). Also Figure 3b appears to show saturation, not monotonic improvement — please align text and figure.","section":"§5.6, Figure 3"},{"comment":"Missing citation context: Point Transformer V3's serialization, Erwin's ball-tree partitioning, and SpiderSolver are compared empirically in Table 8, which is welcome; however the comparison adapts PTv3/SpiderSolver with replaced heads — please report their parameter counts alongside the results (PTv3 'largest configuration' is much larger than 30M) so the comparison is interpretable.","section":"§5.7, Table 8"}],"recommendation":"minor_revision","confidential_remarks":"The strongest baselines here (Transolver-3, GeoTransolver, AB-UPT, LinearNO) are all very recent preprints, some from groups adjacent to the authors' ecosystem; the field is moving fast enough that \"state of the art\" has a short shelf life, and the adopted Transolver-3/AB-UPT numbers should be verified against a common split before the SOTA language is finalized. The REALM and GRaM-competition benchmarks have author-side connections (co-authors on the REALM paper), which is disclosed by the author list but worth the editor noting for benchmark-independence context. None of this affects the technical assessment; the reproduced-baseline evidence is the strongest part of the paper."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a competent systems paper that stitches Hilbert patches, local adaptive projection bases, factorized attention, and cached arbitrary-query decode into one recipe that actually runs on industrial meshes. The distinctive move is not any single ingredient—those are familiar from PTv3, Transolver, Galerkin attention, UPT/GINO—but patch-local function projection with shared token indices plus the factorized space/state attention, and the practical payoff is decoding a full ~8M-point surface from ~100K support tokens without measurable error (Table 9) and much faster than Transolver-3’s tiled path.\n\nWhat it does well: six benchmarks spanning regular grids, irregular multiphysics, and 3D CFD; ablations on ordering, P, M, and mesh scale that match the design story; a clean complexity argument; and a Galerkin-style reading that is interpretive rather than fake theory. Table 2 (reproduced baselines) looks like the cleanest evidence of stable rollouts versus global-slice methods. The efficiency tables are useful if you care about Cd/Cl on full meshes.\n\nSoft spots, in proportion: the industrial SOTA line in the abstract and §5.2 leans on Transolver-3/AB-UPT numbers adopted from their papers while DrivAerML uses the authors’ own 400/50/50 split with no official anchor. They re-ran GeoTransolver for L1/L2 mismatch, so they know cross-paper numbers can lie—yet they did not document the same check for the adopted baselines. Margins are small (e.g. ps 3.35 vs 3.71; Cd R² ties at 0.996). No multi-seed uncertainty anywhere. Residual “SGNO” naming is sloppy. The shared-index “aligned physical states” premise is load-bearing for the factorization and is asserted more than tested. None of that voids the architecture or the internal decode result; it does mean “SOTA on industrial CFD” is not yet bankable.\n\nWho it’s for: people building geometry-flexible PDE/CFD surrogates who need a concrete tokenizer + attention pattern and a full-mesh inference trick. Not for approximation theorists.\n\nI’d send it to referees. Ask for same-split re-runs or released baselines, multi-seed error bars, code, and a toned-down industrial claim until those land. Worth engaging if you work this stack; skim the method and Table 2/9 first, treat Tables 3–4 as provisional.","headline":"Solid engineering architecture for PDE Transformers with a real efficiency win on full-mesh decode; industrial SOTA claims need same-split verification before you trust the leaderboard.","tokens_in":24804,"tokens_out":629,"would_cite":true,"duration_ms":16316,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Projecting physical fields onto adaptive local bases yields Transformer tokens that solve PDEs on general meshes at state-of-the-art accuracy.","keywords":["Physics Transformer","neural operators","PDE prediction","function projection","physics tokens","factorized attention","irregular meshes","CFD surrogates"],"falsifier":"On a multiscale irregular-mesh task, force token indices to be unaligned or replace factorized attention with full attention over all patch-token pairs; if accuracy and rollout stability collapse relative to the reported tables, the alignment premise fails.","tokens_in":24365,"feed_emoji":"⚙️","tokens_out":730,"duration_ms":13679,"temperature":0.7,"pith_summary":"Physical fields are continuous functions sampled on meshes, not words or image patches, so a Transformer needs tokens that respect that functional structure. Physics Transformer partitions any discretization into locality-preserving patches, learns adaptive basis functions inside each patch, and projects the local field onto those bases to form compact physics tokens. Factorized attention then couples the same physical-state index across patches and mixes states inside each patch, after which the tokens decode back to fields at arbitrary query points. Across 2D dynamics, irregular multiphysics, and industrial 3D CFD, the method reports the best predictive accuracy, stable multi-step rollouts, and full-mesh inference from a modest support set with no accuracy loss. The practical claim is that function projection is a workable foundation for Transformer PDE solvers on general geometries.","feed_headline":"Local function projection gives Transformers PDE tokens that win","feed_subtitle":"Patch bases plus factorized attention beat global slices from 2D dynamics to industrial 3D CFD","key_machinery":"Patch-wise physics tokenization: inside each Hilbert-ordered spatial patch a shared generator builds M adaptive basis functions; the local field is projected onto them to yield M physics tokens whose indices stay aligned across patches, enabling factorized space-then-state attention and arbitrary-query de-tokenization.","core_discovery":"Function-projection tokenization inside locality-preserving patches, followed by factorized attention over aligned physics-token indices and within-patch states, produces a general Transformer operator that captures fine-grained physical structure and achieves state-of-the-art field prediction from 2D PDE dynamics through industrial-scale 3D CFD, including stable autoregressive rollouts and accurate full-mesh decoding from cached support tokens.","pith_inferences":["The same local-projection idea could transfer to other continuous-field operators outside fluids, such as elasticity or electromagnetics on unstructured meshes.","If index alignment is imperfect, a light cross-state long-range path or learned index routing might close remaining gaps without restoring full quadratic cost.","Cached-token decoding suggests a natural path to interactive design loops where geometry changes only re-encode a small support set."],"forward_implications":["Function projection can replace global slice aggregation as the default way to tokenize continuous fields for Transformers.","Industrial CFD surrogates can train on ~100K support points and decode full multi-million-point meshes from cached tokens without accuracy loss.","Stable multi-step rollout on stiff multiphysics becomes achievable without geometry-specific architectures.","Complexity drops from quadratic in mesh size to costs linear in points plus quadratic only in patches and tokens per patch."],"fun_headline_variants":["Function projection turns field patches into PDE tokens that win","Physics Transformer: local bases beat global slices on 2D–3D PDEs","Patch-wise projection plus factorized attention leads PDE prediction","Adaptive local bases give Transformers SOTA physical field forecasts","Function-projected tokens capture fine structure from 2D PDEs to 3D CFD"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That a shared basis generator keeps the same token index meaning the same latent physical state in every patch, so factorized attention across those indices is enough and denser coupling is unnecessary.","fun_headline_variants_meta":{"raw":{"variants":["Function projection turns field patches into PDE tokens that win","Physics Transformer: local bases beat global slices on 2D–3D PDEs","Patch-wise projection plus factorized attention leads PDE prediction","Adaptive local bases give Transformers SOTA physical field forecasts","Function-projected tokens capture fine structure from 2D PDEs to 3D CFD"]},"model":"grok-4.5","effort":"low","cost_usd":0.001761,"raw_usage":{"total_tokens":898,"prompt_tokens":798,"num_sources_used":0,"completion_tokens":80,"cost_in_usd_ticks":17608000,"prompt_tokens_details":{"text_tokens":798,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":20,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":798,"tokens_out":80,"duration_ms":2496,"temperature":1.0,"reasoning_tokens":20,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T12:45:46.028465+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a multiscale irregular-mesh task, force token indices to be unaligned or replace factorized attention with full attention over all patch-token pairs; if accuracy and rollout stability collapse relative to the reported tables, the alignment premise fails.","supporting_citations":[],"review_version":1}