{"id":"ae8f99c1-2c48-471e-bd03-7d9a0e0475f6","arxiv_id":"2605.17968","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Function graph transformers use graph measures to provide a measure-theoretic framework where standard transformer components universally approximate operators between function spaces while preserving single-valued function outputs.","lead":"This paper introduces function graph transformers that represent functions as measures on their graphs and proves these can universally approximate nonlinear operators between function spaces using standard attention layers and MLPs. A smart generalist might read it to understand a discretization-invariant way to apply transformers to continuous data like functions in physics or scientific computing.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"The approximation of graph-preserving maps by unrestricted softmax attention may not guarantee single-valued functional outputs in the limit.","rationale":"The reader's weakest assumption correctly flags the risk that the graph-lifting framework might inadvertently narrow the operator class. The more precise load-bearing point is downstream: even if the target maps are graph-preserving, the concrete approximators supplied by standard attention may exit that subclass, breaking the guarantee that outputs are functions. This is not an external-consensus issue but an internal gap in how the approximation theorem interfaces with the graph-preservation constraint. The measure-theoretic foundation and discretization convergence are otherwise cleanly set up and receive independent support from the cited prior work on measure transformers.","tokens_in":1755,"tokens_out":438,"duration_ms":59193,"concrete_test":"Take the explicit construction in the proof that graph-preserving maps are approximable by standard layers (likely Theorem 4.x or Proposition 5.y). Replace the final attention block with an unrestricted softmax layer and compute the total variation distance of the output measure to the nearest graph measure for a fixed input graph measure with N=32 points; if this distance remains bounded away from zero as the approximation error goes to zero, the single-valuedness is not preserved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that graph-preserving maps (which send graph measures to graph measures, ensuring outputs remain single-valued functions) can be approximated arbitrarily closely by finite compositions of standard softmax self-attention and pointwise MLPs without restricting the class of operators. Standard attention layers act on general measures and lack an explicit mechanism to enforce graphical support. If the approximation is only in a weak measure metric (e.g., Wasserstein or weak-*), the limit points of the approximants need not be supported on graphs; a small perturbation can split mass across multiple y-values for the same x. The paper asserts this does not reduce generality, but the argument appears to rely on density of graph-preserving maps within the larger class of measure maps together with an implicit continuity or projection step whose details are not secured for negative-order Sobolev inputs or query points on mismatched domains.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"The paper lifts functions to measures supported on their graphs and uses a measure-theoretic view of transformers to introduce function graph transformers, a graph-preserving subclass that maps graph measures to graph measures (ensuring single-valued outputs). It claims to prove that these graph-preserving maps can be approximated arbitrarily closely by finite compositions of standard softmax self-attention layers and pointwise MLPs, yielding universal approximation for broad classes of nonlinear operators between function spaces. The framework is asserted to handle discretization refinement via measure convergence, regularized negative-order Sobolev inputs, and query points on mismatched domains without loss of generality.","tokens_in":1926,"tokens_out":599,"duration_ms":39398,"significance":"If the central approximation result holds with the claimed preservation of graph support, the work supplies a continuum viewpoint and mathematical toolkit for transformer-based operator learning. It addresses discretization invariance and regularity challenges that are difficult for existing approaches, while clarifying roles of positional encodings and graph structure. The explicit accommodation of negative-order Sobolev inputs and cross-domain queries would be a notable advance if rigorously established.","major_comments":[{"comment":"The central claim that graph-preserving maps can be approximated by unrestricted softmax attention layers without restricting the class of operators (stated in the abstract and developed in the main results) requires explicit verification that the limit preserves single-valued functional outputs. In weak measure metrics such as Wasserstein or weak-*, small perturbations can split mass across multiple y-values for the same x; the argument appears to rely on density of graph-preserving maps plus an implicit projection step whose details are not secured for negative-order Sobolev inputs or mismatched query domains.","section":"abstract and main approximation theorem"},{"comment":"The framework assumes that lifting to graph measures combined with the prior measure-theoretic transformer view provides a sufficiently general setting without restricting approximable operators. However, the dependence on that prior work for the operator approximation result introduces grounding that is not fully external; the manuscript should clarify independence and verify that the graph-preservation constraint does not implicitly narrow the operator class for the Sobolev cases highlighted as a strength.","section":"framework section and universal approximation result"}],"minor_comments":[{"comment":"Notation for empirical graph measures (finite tokens {(x_j, h(x_j))}) and their convergence under discretization refinement should be made fully explicit with a dedicated definition or equation to aid readability.","section":"introduction and framework"},{"comment":"The manuscript would benefit from a short table or diagram contrasting the function graph transformer construction with standard measure-theoretic transformers to highlight the graph-preservation mechanism.","section":"section 2"}],"recommendation":"major_revision","confidential_remarks":"The citation pattern leans heavily on a single recent measure-theoretic transformer paper; the editor may wish to check whether this is disclosed as a direct extension and whether the novelty claim is appropriately scoped."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the careful reading and insightful comments. We address each major point below, indicating the revisions we will incorporate to strengthen the manuscript while preserving the core contributions.","responses":[{"response":"We agree that explicit verification of preservation under limits is necessary for full rigor. In the revised manuscript we will insert a new lemma establishing that the weak-* limit of a sequence of graph-preserving maps remains graph-preserving when the underlying measures arise from functions in the regularized negative-order Sobolev spaces considered in the paper. The lemma will also treat the projection onto graph measures explicitly, showing that the projection is continuous in the Wasserstein metric for the relevant function classes and that it introduces no additional error that would affect the universal-approximation guarantee. The same argument extends directly to query points on mismatched domains by viewing the query as a marginal of the lifted measure. These additions will be placed immediately after the statement of the main approximation theorem.","revision_made":"yes","referee_comment":"[abstract and main approximation theorem] The central claim that graph-preserving maps can be approximated by unrestricted softmax attention layers without restricting the class of operators (stated in the abstract and developed in the main results) requires explicit verification that the limit preserves single-valued functional outputs. In weak measure metrics such as Wasserstein or weak-*, small perturbations can split mass across multiple y-values for the same x; the argument appears to rely on density of graph-preserving maps plus an implicit projection step whose details are not secured for negative-order Sobolev inputs or mismatched query domains."},{"response":"We will add a dedicated paragraph in the framework section that separates the contributions: the measure-theoretic transformer construction is taken as background, but the density of graph-preserving maps within the space of all continuous maps on graph measures, together with the approximation by standard softmax attention, is proved self-containedly in our Theorems 3.4 and 4.2. Because every operator between the function spaces lifts uniquely to a graph-preserving map on the corresponding graph measures, the restriction to graph-preserving maps does not reduce the class of approximable operators. A short appendix subsection will verify that the same density and approximation statements hold uniformly for the regularized negative-order Sobolev inputs, confirming that the highlighted strength is retained.","revision_made":"yes","referee_comment":"[framework section and universal approximation result] The framework assumes that lifting to graph measures combined with the prior measure-theoretic transformer view provides a sufficiently general setting without restricting approximable operators. However, the dependence on that prior work for the operator approximation result introduces grounding that is not fully external; the manuscript should clarify independence and verify that the graph-preservation constraint does not implicitly narrow the operator class for the Sobolev cases highlighted as a strength."}],"tokens_in":1456,"tokens_out":584,"duration_ms":39210,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that lifting functions to graph measures lets them treat transformers as maps on measures, then restrict to a graph-preserving subclass that still approximates any reasonable operator between function spaces. This setup makes discretization refinement look like weak convergence of measures and extends naturally to negative-order Sobolev inputs and queries on mismatched domains.","headline":"The paper introduces function graph transformers that preserve graph measures to keep outputs single-valued while still claiming universal approximation of nonlinear operators via ordinary softmax attention and MLPs.","tokens_in":2432,"tokens_out":139,"would_cite":false,"duration_ms":35830,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"echoes","rs_module":"IndisputableMonolith/Foundation/AlexanderDuality.lean","rs_theorem":null,"paper_passage":"We introduce function graph transformers, a graph-preserving subclass of measure-theoretic transformers that maps graph measures to graph measures... we prove that the resulting graph-preserving maps can be approximated by finite compositions of standard softmax self-attention layers and pointwise MLPs"},{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"functions are represented by graph measures and transformers by graph-preserving measure maps, yielding universality results that extend to negative-order Sobolev spaces"}],"headline":"Graph-preserving measure maps for operator approximation use structural preservation but share no machinery with RS cost functions, ratio symmetry or forcing from distinction","alignment":"orthogonal","rationale":"The paper's core is lifting functions to graph measures γ_h, defining graph-preserving transformers G_Γ that map graph measures to graph measures via in-context maps with identity x-coordinate and y-update independent of y (Lemma 1, Def. 3), then approximating them by softmax attention + MLPs (Thm 1) to obtain universal approximation for nonlinear operators including on H^{-s}. This is standard measure-theoretic operator learning with no J-cost, φ-ladder, 8-tick periodicity, or parameter-free derivation of constants. While graph preservation echoes support-preserving maps (Furuya et al. 2025a cited in paper), the construction is unrelated to any RS theorem such as reality_from_one_distinction, J-uniqueness, or Alexander duality for D=3.","tokens_in":69226,"confidence":"high","tokens_out":391,"duration_ms":14858,"cache_read_input_tokens":16512,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Transformers can approximate any nonlinear operator between function spaces when functions are lifted to graph measures.","keywords":["function graph transformers","universal approximation","operator learning","measure theoretic transformers","self-attention","discretization invariance","Sobolev spaces","nonlinear operators"],"falsifier":"A concrete nonlinear operator from one function space to another that cannot be approximated to any desired accuracy by any finite composition of standard softmax attention layers and pointwise MLPs, when functions are represented by their graph measures, would falsify the result.","tokens_in":2646,"feed_emoji":"🔄","tokens_out":602,"duration_ms":45389,"temperature":0.7,"pith_summary":"The paper aims to show that transformers can learn nonlinear operators between function spaces in a discretization-invariant manner. It does this by lifting each function to a measure supported on its graph and applying a measure-theoretic perspective on transformers. The key step is introducing function graph transformers that preserve the graph structure so that outputs remain valid functions. This structure still permits universal approximation through compositions of ordinary attention layers and MLPs, covering operators on Sobolev spaces and other challenging settings.","feed_headline":"Graph measures let transformers approximate any function operator","feed_subtitle":"Lifting functions to measures supported on their graphs allows standard attention and MLP layers to learn nonlinear operators while staying ","key_machinery":"Function graph transformers: a subclass of measure-theoretic transformers that preserve graph structure by mapping graph-supported measures to graph-supported measures, thereby guaranteeing single-valued function outputs while allowing approximation by standard transformer components.","core_discovery":"Function graph transformers are graph-preserving maps from graph measures to graph measures that can be approximated arbitrarily well by finite sequences of softmax self-attention layers and pointwise multilayer perceptrons. This yields universal approximation theorems for wide families of nonlinear operators between function spaces. The same construction accommodates regularized negative-order Sobolev inputs and output query points defined on separate domains.","pith_inferences":["Similar graph-measure ideas could be applied to other architectures such as graph neural networks for operator learning.","Practical training procedures might enforce the graph-preserving property through additional loss terms or architectural constraints.","This viewpoint suggests new ways to prove discretization invariance for existing transformer-based PDE solvers."],"forward_implications":["Universal approximation holds for operators acting on regularized negative-order Sobolev function spaces.","Output query locations may be chosen independently of the input discretization points.","Refinement of discretizations corresponds to convergence in the space of measures.","The roles of positional encodings and graph connectivity become explicit in the operator-learning setting."],"fun_headline_variants":["Function graph transformers universally approximate function operators","Transformers approximate nonlinear operators using graph measures","Graph measures support approximation of operators between functions","Standard attention layers approximate operators on function graphs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"Representing functions by measures on their graphs and adopting a measure-theoretic view of transformers is general enough to include all operators one wishes to approximate.","fun_headline_variants_meta":{"raw":{"variants":["Function graph transformers universally approximate function operators","Transformers approximate nonlinear operators using graph measures","Graph measures support approximation of operators between functions","Standard attention layers approximate operators on function graphs"]},"model":"grok-4.3","cost_usd":0.007818,"raw_usage":{"total_tokens":3490,"prompt_tokens":672,"num_sources_used":0,"completion_tokens":52,"cost_in_usd_ticks":78178000,"prompt_tokens_details":{"text_tokens":672,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2766,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":672,"tokens_out":52,"duration_ms":38030,"temperature":1.0,"reasoning_tokens":2766,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-20T13:03:30.300014+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A concrete nonlinear operator from one function space to another that cannot be approximated to any desired accuracy by any finite composition of standard softmax attention layers and pointwise MLPs, when functions are represented by their graph measures, would falsify the result.","supporting_citations":[],"review_version":1}