{"id":"f3f58286-13ac-4b33-9bfe-bfe6a1051ce3","arxiv_id":"2506.08226","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Mondrian applies transformer attention to subdomain-restricted functions, decoupling the model from the grid resolution.","lead":"Mondrian is a transformer architecture for learning PDE solution operators that divides the domain into subdomains and computes attention over functions restricted to those subdomains. It lets transformer-based neural operators handle higher-resolution inputs than they were trained on, without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The L2 inner product in Algorithm 1 has no stated domain of integration; with disjoint subdomain supports it vanishes off-diagonally, so cross-subdomain attention would reduce to learned position bias and the central global-interaction claim is unsupported as written.","rationale":"A good-faith reading is that the paper's contribution is a resolution-agnostic transformer operator in which attention is computed between subdomain-restricted functions via L2 inner products, giving content-based global interactions while keeping sequence length fixed. For this central claim to hold, the inner product must be well-defined and generally nonzero across different subdomains. The paper does not provide that definition. The only extension operator introduced in Section 2.2 is zero extension, and under zero extension every off-diagonal inner product in Algorithm 1 vanishes identically. The learned bias B could still mix subdomains, but that is a fixed relative-position bias, not function-content attention; the abstract's claim about softmax-based inner products over functions would then be false. A charitable reading using Ω_i≃Ω_j as affine equivalence could rescue the construction by pulling all subdomain functions back to a reference domain, but the paper never states this pullback or the quadrature used, so the algorithm cannot be reimplemented uniquely from the text. I also note a secondary but concrete issue: Algorithm 3's neighborhood attention sets non-neighbor scores to 0 rather than -inf, and since softmax treats 0 as an ordinary score, the neighborhood variant does not actually restrict attention to nearby subdomains; this reinforces that the attention definitions need correction. The empirical tables do show modest error growth across resolutions and the Mixture Operator ablations are informative, so the paper has promising pieces, but the central mechanism is currently underspecified. The reader's CONDITIONAL verdict is appropriate, and the condition should require a precise definition of the inner product/reference domain plus code or a minimal numerical demonstration of nonzero off-diagonal scores. My stress-test therefore leaves the verdict unchanged.","tokens_in":20353,"tokens_out":11436,"duration_ms":142002,"concrete_test":"Obtain or request the released implementation and instrument Algorithm 1 to print the raw score matrix S for a fixed input on a two-subdomain decomposition. If the code computes ⟨Q_k,K_j⟩ by zero-extending to Ω or by aligning global grid points, S_{1,2} will be 0; then confirm the consequence by retraining the ViT-NO-SepMO Allen-Cahn model with all off-diagonal inner products removed (score = B only) and comparing the 32×32 MAE to Table 4, since an unchanged MAE shows cross-subdomain content attention is not load-bearing. If the implementation instead pulls functions back to a reference subdomain and S_{1,2}≠0, the authors should specify that construction explicitly and demonstrate the nonzero scores by numerical quadrature.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines S_{k,j}=⟨Q_k,K_j⟩ and states 'We use the L2 inner product,' but Q_k and K_j are outputs of I_QKV applied to restrictions v_{Ω_k} and v_{Ω_j}, so the paper never specifies the domain on which these functions live or the domain of integration. If Q_k and K_j are functions on their own subdomains, L2(Ω_k) and L2(Ω_j) are different spaces and the pairing is undefined. If they are zero-extended to Ω, using the only extension operator defined in Section 2.2, then their supports are disjoint and S_{k,j}=0 for k≠j. The off-diagonal attention would then depend only on the learned relative-position bias B introduced in Section 3.2, not on function content, so the architecture would not realize the claimed softmax-based inner products over functions for global interaction. The notation Ω_i≃Ω_j hints that a pullback to a common reference subdomain might be intended, but no such construction, quadrature rule, or shared basis is given, and Algorithm 1 as written is either ill-posed or degenerate. This is load-bearing because it is the mechanism by which Mondrian claims content-based global interactions and resolution-independent sequence length.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Mondrian, a family of transformer architectures for PDE operator learning. The input function is decomposed into non-overlapping subdomains, and the transformer sequence is a set of subdomain-restricted functions rather than point vectors. Attention between subdomains is defined through L2 inner products of query and key functions, with optional relative position bias; local variants use windowed and neighborhood attention. Subdomain-level operators, including a new Mixture Operator, replace linear layers. The authors evaluate on Allen-Cahn and 2D Navier-Stokes, reporting generalization from 32x32 training to 64x64 and 128x128 test resolutions and comparing with FNO, FFNO, Galerkin Transformer, and FactFormer.","tokens_in":20645,"tokens_out":8149,"duration_ms":94561,"significance":"The architectural idea is attractive: it keeps exact softmax attention while making the sequence length depend on the number of subdomains rather than the discretization, and it provides a modular set of subdomain operators. The experimental protocol is careful, with held-out resolutions, validation-based hyperparameter selection, parameter counts, and extensive ablations, and the paper is transparent about limitations such as spectral bias and zero-shot superresolution. However, the central definition of the score inner product is ambiguous, and under the only extension operator defined in the paper the cross-subdomain attention scores vanish. This gap directly affects the claimed mechanism for global interaction and must be repaired before the architectural claims can be assessed.","major_comments":[{"comment":"The score definition S_{k,j}=<Q_k,K_j> is either undefined or degenerate because no common domain for Q_k and K_j is specified. The only function-space convention introduced in Section 2.2 uses restrictions to subdomains and zero extensions E_{Omega_i} to Omega. Under that convention, Q_k and K_j with k != j have disjoint supports, so <Q_k,K_j>_{L2(Omega)}=0, softmax attention is diagonal, and cross-subdomain content communication vanishes; the off-diagonal scores would then reduce to the learned position bias B described in Section 3.2, which the paper states is used in the experiments. The notation Omega_i ≃ Omega_j suggests that a pullback to a reference subdomain is intended, but no such construction, shared basis, or quadrature is given. This is load-bearing because it is the mechanism for the abstract's claim of attention across subdomains via softmax-based inner products over functions.","section":"Section 3.2, Algorithm 1 (line 3)"},{"comment":"The recombination step u_{Omega_i}=softmax(tau S)_i V is also not well-defined as written: each V_j is a function on Omega_j or its zero extension, so forming a weighted sum over j and then applying E_{Omega_i} from Omega_i to Omega either requires adding functions on different domains or discards the V_j contributions outside Omega_i. A consistent construction, such as pullback to a common reference subdomain followed by pushforward, must be supplied. This is not a notational issue because the output function of the architecture is defined through this step.","section":"Algorithm 1, lines 4-5"},{"comment":"The claim that Mondrian 'matches state-of-the-art accuracy' is stated more strongly than the evidence. In Table 4, the best Mondrian variant has MAE 3.58e-3 at 32x32 and 8.12e-3 at 128x128, whereas FFNO has 4.35e-3 and 5.11e-3; at the highest resolution FFNO is both more accurate and more stable across resolutions. In Table 5, the best Mondrian model (Swin-NO-MO) has MAE 7.07e-2 compared with FFNO's 5.05e-2. The results show competitive performance and nontrivial resolution transfer, but the stated match to state-of-the-art accuracy should be qualified accordingly.","section":"Abstract, Section 4.3, Section 4.4, Tables 4 and 5"}],"minor_comments":[{"comment":"The compressed notation with the domain 'assumed to be clear from the context' is exactly what makes Algorithm 1 ambiguous; a concrete domain should be attached to every function in Algorithm 1.","section":"Section 2.4"},{"comment":"The expression 'softmax(tau S)_i V' should include the subscript j on V and the summation over j to match standard attention; as written it is unclear what is being summed.","section":"Section 3.2"},{"comment":"The range for gamma is stated as [1e-4, 5e-3] in Section 4.3 but as [1e-5, 5e-3] in Appendix C.1; these should be reconciled.","section":"Section 4.3 and Appendix C.1"},{"comment":"Swin-NO-SpecConv reports a two-step MSE of 5.25e-1 and a rollout MSE of 1.26e-1; since rollout error compounds over multiple steps, this inversion is unexpected and warrants verification.","section":"Appendix E.2, Table 7"},{"comment":"The comparison with Continuum Attention [6] would benefit from an explicit statement of whether that work's score inner product uses a common reference domain; the current phrasing gives no basis for the claimed difference.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not ready in its current form because Algorithm 1 is internally inconsistent. The fix is likely local, by introducing a reference subdomain and pullback maps, so I do not recommend rejection, but the authors must also re-examine the empirical headline after the construction is clarified. The overlap with Continuum Attention [6] is nontrivial, and a precise technical comparison is needed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely nice: treat subdomain-restricted functions as tokens, replace linear layers with integral operators, and adapt ViT/Swin attention to operate over function sequences. The Mixture Operator and the shifted-window variants are real additions, and the systematic ablation across subdomain operators is the kind of work the field needs. I also credit the authors for being candid about attention's low-pass behavior in the Navier-Stokes rollouts and for acknowledging that zero-shot super-resolution remains hard.\n\nThe soft spot is exactly where the reader's report points. Algorithm 1 defines S_{k,j} = <Q_k, K_j> with the L2 inner product, but Q_k lives on Ω_k and K_j on Ω_j. Unless the authors specify a common reference domain, a pullback, or some other shared function space, that pairing is either undefined or, if zero-extended to Ω, identically zero for k≠j. The paper's notation \"Ω_i ≃ Ω_j\" hints at a diffeomorphism but never lays out the change of variables or quadrature. So as written, the off-diagonal attention scores reduce to the learned position bias B, and the model stops being content-based across subdomains. That is a load-bearing flaw, not a cosmetic one.\n\nA second issue is that the empirical claims are a bit ahead of the tables. On Allen-Cahn, the best Mondrian models do beat the baselines at 32x32, but the MAE roughly doubles at 128x128; calling that \"low error\" is generous. On Navier-Stokes, Swin-NO-MO is competitive with FFNO on rollout, but ViT-NO is markedly worse. The abstract's \"strong performance\" overstates the picture. Also, no code or data is released, which makes it hard to check the inner product ambiguity in practice.\n\nMinor note: the related-work discussion is fair, and the paper correctly distinguishes itself from continuum attention and masked autoencoder analysis.\n\nWho is this for? Researchers building transformer-based neural operators and anyone thinking about resolution-agnostic attention. The paper deserves a serious referee—the ideas are worth engaging—but that referee should demand a precise definition of the inner product, a demonstration that cross-subdomain scores are nonzero and content-dependent, and ideally artifact release. If the inner product turns out to be degenerate, the architecture still works via position bias, but then the paper needs to reframe its contribution accordingly.","headline":"Useful architecture paper, but the cross-subdomain inner product is underspecified and likely vanishes, so the central attention-over-functions claim needs a fix before it can stand.","tokens_in":21137,"tokens_out":1738,"would_cite":false,"duration_ms":22825,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65M55"],"pacs":[],"model":"deepseek-v4-flash","headline":"Mondrian computes exact softmax attention over subdomain-restricted functions, decoupling transformer operator learning from grid resolution and enabling PDE models trained at 32×32 to run at 128×128 without retraining.","keywords":["operator learning","neural operators","transformer attention","domain decomposition","softmax attention over functions","resolution generalization","Allen-Cahn equation","Navier-Stokes turbulence"],"falsifier":"Compute the score matrix from Algorithm 1 for actual trained queries and keys: if $S_{k,j}$ is evaluated as the standard $L^2$ inner product over $\\Omega$ (or over the union of subdomains), all off-diagonal entries with $k \\neq j$ are exactly zero for functions with disjoint support, so the model would reduce to independent per-subdomain processing. Checking whether off-diagonal scores are nonzero, and over what domain the integration runs, settles whether the claimed global attention is real.","tokens_in":20138,"feed_emoji":"🧩","tokens_out":8174,"duration_ms":85548,"temperature":0.7,"pith_summary":"The paper introduces Mondrian, a transformer architecture for operator learning that decomposes the input domain into non-overlapping subdomains and treats the input as a sequence of subdomain-restricted functions rather than a sequence of grid points. Its central claim is that this function-level attention decouples the model from discretization: subdomain sizes are fixed by physical domain size, not by resolution, so a model trained at $32\\times32$ can be evaluated at $128\\times128$ without retraining. On Allen-Cahn and Navier-Stokes benchmarks, Mondrian matches the best baselines at training resolution and keeps stable error when the test grid is refined. The authors argue this makes exact softmax attention viable for PDE operator learning instead of relying on subquadratic attention approximations.","feed_headline":"PDE transformer decouples attention from grid resolution","feed_subtitle":"Mondrian treats inputs as functions on subdomains, so a 32×32-trained model applies to 128×128 without retraining.","key_machinery":"The load-bearing object is subdomain self-attention over functions (Algorithm 1). Instead of patching points as vision transformers do, it uses a domain decomposition $D_s(\\Omega)$ with restriction and zero-extension operators, computes queries, keys, and values as functions via neural operators, and replaces the vector dot product $QK^T$ with the $L^2$ inner product $\\langle Q_k,K_j\\rangle$ to build a finite score matrix. A second component is the mixture operator, whose kernel $\\kappa(x,y)=\\sum_i M_i C_i(x,y)$ mixes learned matrices with coefficients from a small network; the paper shows this subdomain operator is the choice that stays accurate as resolution grows.","core_discovery":"Mondrian's central claim is that attention can be computed over functions rather than vectors: decompose $\\Omega$ into $s$ non-overlapping subdomains, map each restricted input through a neural operator $I_{QKV}$ to query, key, and value functions, form the score matrix $S_{k,j}=\\langle Q_k,K_j\\rangle$ with the $L^2$ inner product, and recompose softmax-weighted values across subdomains. Because the subdomain partition is fixed in physical space, the sequence length $s$ is independent of how finely the function is sampled, which the paper identifies as the mechanism behind resolution generalization. Experimentally, the mixture subdomain operator—a kernel built as a learned mixture of trainable matrices—gives the most stable resolution scaling, and a Swin-style shifted-window version performs best on turbulent Navier-Stokes rollouts.","pith_inferences":["The paper does not specify the integration domain for $\\langle Q_k,K_j\\rangle$; if cross-subdomain terms are implemented as zero-extension inner products over $\\Omega$, the model's resolution transfer becomes a claim about the kernel's sampling behavior that the mixture operator's learned coefficients could be tested against directly.","Because vanishing off-diagonal scores would reduce the method to per-subdomain processing, a cheap check is to permute subdomain indices at inference: truly global attention would change outputs, while an effectively diagonal score matrix would not.","The localized cross-attention idea for grid-to-grid mapping could be lifted out as a standalone interpolation or coarsening operator; measuring its error against standard interpolation on an irregular grid would quantify how much function-level attention contributes beyond geometry-aware pooling."],"forward_implications":["Exact softmax attention over subdomains stays quadratic in the number of subdomains rather than grid points, so finer discretizations no longer increase attention sequence length.","A Mondrian model trained only on $32\\times32$ Allen-Cahn data can be evaluated at $64\\times64$ and $128\\times128$ without retraining, with the best variants keeping MAE below $10^{-2}$ at the finest resolution.","Swin-style shifted-window attention pays off on turbulent Navier-Stokes: the Swin-NO with the mixture operator reaches the lowest two-step and rollout errors among the compared models.","The choice of subdomain operator is decisive: mixture and separable-mixture kernels beat spectral convolution, low-rank kernels, and in-subdomain attention, with the separable variant doing so at roughly five times fewer parameters."],"supporting_citations":[{"why":"Closest prior work: also uses restriction and extension operators for attention over functions; Mondrian builds on and differentiates from this construction.","marker":"[6]"},{"why":"Provides the Galerkin-style linear attention baseline whose limitations motivate exact softmax attention over subdomains.","marker":"[7]"},{"why":"Supplies the Fourier neural operator as both a baseline and the spectral convolution subdomain operator option.","marker":"[34]"},{"why":"Supplies shifted windows and relative position bias, adapted into windowed and neighborhood function attention.","marker":"[40]"},{"why":"Provides the kernel integral operator formulation and function-space notation the whole method is built on.","marker":"[33]"},{"why":"Supplies the restriction and extension operators and decomposition vocabulary from domain decomposition theory.","marker":"[16]"},{"why":"Used to argue that exact softmax attention can be run efficiently on hardware, motivating the design choice.","marker":"[13]"},{"why":"FactFormer is the most competitive transformer baseline on resolution transfer, providing the accuracy comparison point.","marker":"[37]"},{"why":"Factorized Fourier Neural Operator is the strongest non-transformer baseline on Navier-Stokes rollouts.","marker":"[52]"}],"fun_headline_variants":["Mondrian: attention over subdomains, not grid points","PDE modeling via function-space attention, resolution-free","Transformer operators that scale to any grid resolution","Domain-decomposed attention yields resolution-independent PDEs","Mondrian: attention on functions for scalable neural operators"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the score $S_{k,j}=\\langle Q_k,K_j\\rangle$ being a meaningful, nonzero inner product between functions defined on different, non-overlapping subdomains; the paper never states the integration domain or function space, and with the usual $L^2$ inner product over the whole domain those cross-subdomain scores vanish, which would collapse attention to a per-subdomain diagonal and remove the cross-subdomain communication the method relies on.","fun_headline_variants_meta":{"raw":{"variants":["Mondrian: attention over subdomains, not grid points","PDE modeling via function-space attention, resolution-free","Transformer operators that scale to any grid resolution","Domain-decomposed attention yields resolution-independent PDEs","Mondrian: attention on functions for scalable neural operators"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000208,"raw_usage":{"total_tokens":1378,"prompt_tokens":893,"completion_tokens":485,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":407}},"tokens_in":509,"tokens_out":485,"duration_ms":4757,"temperature":1.0,"reasoning_tokens":407,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:17:14.141891+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the score matrix from Algorithm 1 for actual trained queries and keys: if $S_{k,j}$ is evaluated as the standard $L^2$ inner product over $\\Omega$ (or over the union of subdomains), all off-diagonal entries with $k \\neq j$ are exactly zero for functions with disjoint support, so the model would reduce to independent per-subdomain processing. Checking whether off-diagonal scores are nonzero, and over what domain the integration runs, settles whether the claimed global attention is real.","supporting_citations":[{"cited_title":"Calvello, N","cited_arxiv_id":null,"evidence_quote":"Closest prior work: also uses restriction and extension operators for attention over functions; Mondrian builds on and differentiates from this construction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Galerkin-style linear attention baseline whose limitations motivate exact softmax attention over subdomains."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Fourier neural operator as both a baseline and the spectral convolution subdomain operator option."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies shifted windows and relative position bias, adapted into windowed and neighborhood function attention."},{"cited_title":"Dolean, P","cited_arxiv_id":null,"evidence_quote":"Supplies the restriction and extension operators and decomposition vocabulary from domain decomposition theory."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Used to argue that exact softmax attention can be run efficiently on hardware, motivating the design choice."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FactFormer is the most competitive transformer baseline on resolution transfer, providing the accuracy comparison point."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Factorized Fourier Neural Operator is the strongest non-transformer baseline on Navier-Stokes rollouts."}],"review_version":1}