{"id":"5711209a-82bc-4a2b-bedd-a3d9b8e03829","arxiv_id":"2505.02974","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PLAID provides a CGNS-based data model, an open-source library, six new simulation datasets, and Hugging Face leaderboards for machine-learning surrogates.","lead":"PLAID is a new data format, software library, and collection of physics-simulation datasets aimed at machine learning. It is worth reading because standardizing how simulation data is stored and shared is a practical bottleneck for AI-driven engineering.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PLAID's central 'wide range' claim is supported only by API examples and six datasets that omit time-dependent remeshing and mixed-element cases; the schema-level expressiveness remains unverified.","rationale":"Agreeing with the reader's weakest assumption. The paper's contribution is primarily a data layer; the most load-bearing sentence is the claim of generic, extensible support for time-dependent problems, remeshing, mixed-element meshes, tagging, multiple dimensions and topologies. What must be true for that claim to hold is that the underlying CGNS mapping plus the plaid API can represent arbitrary combinations of these features. Nothing in the paper proves that; the six datasets are existence proofs for their particular feature subsets. The absence of a mixed-element dataset and the fixed-mesh-within-sample time series make the gap concrete rather than hypothetical. I nevertheless do not move the verdict: the reader already set CONDITIONAL, and the required condition is exactly a synthetic roundtrip demonstration or a scoping statement. I would also keep the other reader conditions (training-code release, seeds and uncertainty estimates, consistent repository URLs) since they concern reproducibility of the benchmark half of the claim. The issue is evidentiary, not a question of author credibility.","tokens_in":23584,"tokens_out":4240,"duration_ms":49126,"concrete_test":"Create a synthetic PLAID sample that combines features absent from the released collection: an unstructured 3D mesh with mixed element types (hexahedra, tetrahedra, prisms), a second zone of different spatial dimension (e.g., a 1D surface embedded in 2D), node and element tags, fields at Vertex and CellCenter locations, and time-dependent remeshing between timesteps (changed connectivity plus element birth/death). Write it with the plaid library, read it back, and require exact roundtrip equality of meshes, fields, tags, and times. If this requires schema extensions or loses any data, the 'supports wide range' claim needs to be scoped down; if it roundtrips, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition for the paper's central claim is that the PLAID/CGNS schema actually expresses the full advertised range—time-dependent remeshing, mixed-element topology, mixed-dimensional embeddings, and fields changing location or availability over time—without user extension. Section 3 gives API-level evidence (get_field with time and location, get_mesh with links), but the released datasets in Section 4 exercise only a subset: all structural and CFD meshes are single-element-type (tri or quad), the only time-dependent dataset (2D_ElPlDynamics) keeps a fixed mesh across timesteps within each sample, and no dataset combines remeshing with mixed element types or changing field topology. The reader's weakest assumption is the same one I would flag: 'unified and extensible' is asserted rather than demonstrated for the combinations that motivated the work. This is a correctness-risk issue for the scope of the claim, not an internal inconsistency: CGNS is a mature standard, and the library may well roundtrip such cases, but the paper provides no evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PLAID, a datamodel built on CGNS for representing and sharing physics-simulation datasets for machine learning, together with a Python library, six released datasets (structural mechanics and CFD, plus AirfRANS variants), and benchmark results from several ML methods (MeshGraphNets, MMGP, Vi-Transformer/Augur, DAFNO, MARIO). The artifacts are publicly released on GitLab, Zenodo, and Hugging Face, with online leaderboards. The paper claims the datamodel supports time-dependent problems, remeshing, mixed-element unstructured meshes, node/element tagging, and multiple spatial dimensions and topologies, and that it 'preserves the full complexity' of simulation data.","tokens_in":23788,"tokens_out":4712,"duration_ms":47739,"significance":"If the expressiveness claims are substantiated, PLAID would fill a real gap: existing physics-ML datasets are fragmented and often tied to narrow formats, and a CGNS-based standard with streaming support via Hugging Face is a credible route to community benchmarks. Concrete strengths include the released artifacts (datasets with documented splits, evaluation code, online leaderboards), the use of a mature standard (CGNS), and the breadth of physical settings (nonlinear structural mechanics, RANS CFD). The benchmark results are useful as reference numbers, even though the paper does not claim state-of-the-art performance. The principal weakness is the gap between the headline claim ('full complexity', 'wide range of use cases') and the evidence: the released datasets exercise only a subset of the advertised features, and no schema-level proof or stress-test is provided for the unexercised combinations. The paper also relies partly on in-house or affiliated baselines for benchmarking, which should be disclosed more prominently.","major_comments":[{"comment":"The paper asserts in Section 3 that the PLAID/CGNS model supports time-dependent remeshing, mixed-element unstructured meshes, and fields that change over time, and the abstract claims it 'preserves the full complexity' of simulation data. However, none of the released datasets exercise these features: every mesh in Table 2 is single-element-type (tri or quad), the only time-dependent dataset (2D_ElPlDynamics) is explicitly described in Appendix B.3 as having a mesh that stays constant through the time sequence, and no dataset contains fields that appear or disappear over time. The evidence for the central expressiveness claim is therefore only API-level, not schema-level or dataset-level. Please either provide a stronger demonstration (for example, a synthetic dataset or round-trip tests covering time-dependent remeshing, mixed elements, and field-topology changes) or re-scope the claim to what the released data actually show.","section":"Section 3 and Table 2"},{"comment":"The abstract and Section 3 claim that the datamodel 'preserves the full complexity of simulation data', but several released datasets deliberately reduce complexity: Rotor37 'only keeps the steady-state solution at the boundary of the blade' (Section 4.2.1), 2D_profile 'only keeps the steady-state solution on a zone cropped close to the profile' (Section 4.2.2), and the AirfRANS variants are provided in original, clipped, and remeshed forms (Section 4.2.4). These are appropriate benchmark choices, but the phrase 'full complexity' should be qualified to refer to the datamodel's representational capacity rather than to the contents of the released datasets.","section":"Sections 4.2.1, 4.2.2, 4.2.4"},{"comment":"The benchmark section does not provide uncertainty estimates (for example, multiple seeds) and does not apply a uniform set of methods across all datasets: DAFNO is evaluated only on 2D_ElPlDynamics, MMGP is not applied to 2D_ElPlDynamics or 2D_MultiScHypEl (as stated in Appendix A.2.2), and no method is evaluated on all six datasets. I do not view this as fatal, since the paper explicitly labels the results as non-exhaustive and non-definitive, but the main text should state these coverage limits alongside Table 4 rather than leaving them entirely to the appendices.","section":"Section 5.1 and Table 4"}],"minor_comments":[{"comment":"There are several typos and template artifacts, for example 'fiels' in Table 2, 'mush be specified' in Appendix B.3, 'tentatives' in Appendix C, and the first page still contains 'Preprint. Under review.' and 'arXiv:2505.02974v2' formatting; a careful copyedit is needed.","section":"General"},{"comment":"The repository URL is inconsistent: the abstract says github.com/PLAID-lib/plaid, while Section 1 and reference [64] say gitlab.com/drti/plaid; the authors should state the canonical location.","section":"Abstract/Introduction"},{"comment":"The field RRMSE normalizes per sample by the maximum reference value rather than by the L2 norm; this is a legitimate choice, but it should be justified because it makes the metric sensitive to outlier nodes in the reference field.","section":"Section 5.2"},{"comment":"The sentence 'In addition to the six original datasets, we provide three variants of AirfRANS in PLAID format' counts the six PLAID datasets plus AirfRANS variants, but the wording is ambiguous about whether AirfRANS itself is one of the 'six original datasets' or an additional seventh; please clarify.","section":"Section 4.2.4"}],"recommendation":"major_revision","confidential_remarks":"The benchmark section leans on models from the same or affiliated groups: MMGP is from the same team, Augur is a commercial product from an affiliated company, and MARIO is from a co-author. This is not a basis for rejection, but the editor may want the authors to add at least one independent external baseline or an explicit conflict-of-interest statement. The gap between the claimed 'full complexity' and the released dataset contents is likely to draw strong reviewer criticism, so the revision should focus on demonstration or re-scoping of that claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. First, PLAID is worth taking seriously: six new simulation datasets with real industrial flavor, a public library, and working Hugging Face benchmarks. Second, the paper's central claim—that the CGNS-based model preserves the full complexity of heterogeneous simulation data—is asserted rather than shown, and the released datasets don't exercise the hard combinations. That mismatch is the main thing to fix.\n\nWhat's actually new: the six core datasets. Tensile2d and 2D_MultiScHypEl give structural mechanics with geometry and topology variability that PDEBench and Well don't cover; Rotor37 and VKI-LS59 add turbomachinery and internal aerodynamics with variable blade shapes; 2D_ElPlDynamics is time-dependent with element erosion. The AirfRANS variants are conversions of an existing dataset, but the conversions themselves are useful, especially the remeshed version. The library is shipped, and the Hugging Face integration with hidden test splits is a credible attempt at community benchmarking. The MMGP, Augur, and MARIO baselines are mostly from the authors' own circle, but the numbers are reported in enough detail to be reproducible, and the paper clearly labels them as non-exhaustive.\n\nSoft spots, in order of importance. The 'wide range' claim in the abstract and Section 3—'time-dependent problems, remeshing, mixed-element unstructured meshes'—is supported only by API examples. The one time-dependent dataset keeps its mesh fixed per sample; no released sample combines remeshing with mixed element types or fields changing topology over time. CGNS is a mature standard, so the schema may well handle these cases, but the paper doesn't demonstrate it. That's a scope-overstatement, not a fatal flaw, but it should be fixed by either adding a schema-level example or trimming the claim to what the datasets show.\n\nSecond, benchmark uncertainty. There are no seeds, no repeated runs, no error bars. For a benchmark paper, that's a real limitation, especially when the differences between methods are sometimes small (e.g., Rotor37 scalars). Third, the method coverage is uneven: MMGP is absent on the topology-varying structural dataset and DAFNO appears only on the dynamics dataset. The paper acknowledges this, but it means the 'total_error' column isn't a fair cross-dataset comparison.\n\nWho's this for? Anyone building or using mesh-based ML surrogates in structural mechanics or CFD. The datasets fill gaps. I'd send it to peer review with a request to address the scope claim and benchmark reproducibility. The artifacts are public and the work is honest.","headline":"PLAID is a genuine dataset-and-library contribution whose 'full complexity' claim outruns the evidence; the artifacts deserve review, but the paper needs to either demonstrate schema expressiveness or soften the claim.","tokens_in":24443,"tokens_out":2428,"would_cite":true,"duration_ms":24487,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces PLAID, a data model that stores heterogeneous physics-simulation datasets—meshes, remeshing, mixed element types, time steps, tags—and releases six benchmark datasets plus baselines to support it.","keywords":["PLAID","physics simulation datasets","scientific machine learning","mesh-based surrogates","data model standard","benchmarking","structural mechanics","computational fluid dynamics"],"falsifier":"Attempt to encode a single simulation that simultaneously has non-conforming mixed-dimensional interfaces, topology change between time steps, and fields defined on cell centers that appear and disappear; if the PLAID library or schema requires modification or loses information, the generality claim is overstated.","tokens_in":23380,"feed_emoji":"🗂️","tokens_out":7454,"duration_ms":80740,"temperature":0.7,"pith_summary":"PLAID is a proposed answer to a missing piece of scientific machine learning: a single data model that can store simulation datasets with all their messiness—unstructured meshes that change from sample to sample and even between time steps, mixed element types, node or element tags, and fields living on different parts of the geometry. The paper argues that this one standard, implemented as a software library, can cover both structural mechanics and computational fluid dynamics, and backs the claim by releasing six original datasets plus three variants of an existing airfoil benchmark. The would-be payoff is that researchers stop writing bespoke converters and instead train, test, and compare surrogate models on shared benchmarks under realistic geometric variability. The paper also supplies baseline numbers from several learning methods and an open leaderboard structure so the community can keep updating those comparisons.","feed_headline":"PLAID gives ML one data model for varied physics simulations","feed_subtitle":"Six open datasets in mechanics and CFD plus public benchmarks let models be compared on realistic, heterogeneous cases.","key_machinery":"The central object is the PLAID sample, a tree of CGNS bases and zones: each zone stores a mesh (nodes, element connectivity, and optional node or element tags) and named fields at a given location, such as a vertex, cell center, or face center, with an explicit time coordinate. The work it does is to make geometry and solution fields first-class, queryable objects, so heterogeneous simulation outputs become homogeneous for machine-learning pipelines. Default argument handling in the library makes simple cases, such as one mesh with one time step and vertex fields, accessible without boilerplate, while explicit zone, location, and time arguments unlock mixed and time-varying cases.","core_discovery":"PLAID formalizes each simulation sample as a hierarchy of named zones inside a scientific data file; each zone holds its own mesh and fields, which can be addressed by location (for example vertices, cell centers, or face centers) and by time step. Meshes and fields may appear, disappear, or change across time steps, allowing remeshing and erosion; repeated meshes can be linked rather than copied; and different zones can live in different spatial dimensions, as in a two-dimensional flow field with a one-dimensional blade surface. The paper claims that this schema, together with a read/write library and human-readable sidecar files, is sufficiently generic and extensible for heterogeneous physics-learning datasets, and it validates the claim by publishing six new datasets and three re-encoded variants of an existing CFD benchmark, reporting relative-root-mean-square-error benchmarks for several representative model families.","pith_inferences":["If the format is adopted beyond this collection, its real test is whether future datasets tick the same boxes without schema changes; the paper's roadmap does not demonstrate this, so adoption is an open empirical question.","The benchmark ranking pattern, with morphing-based and implicit-neural-field approaches leading on steady-state datasets and a Fourier operator method on the time-dependent one, suggests that no single architecture is likely to sweep the collection, which makes the leaderboard useful for method selection but also puts weight on how RRMSEs are aggregated across fields and scalars.","A testable extension would be to convert an existing popular benchmark not designed for remeshing into PLAID and measure whether models transfer across the original and the PLAID encodings; if results shift materially, the format is not neutral."],"forward_implications":["A dataset encoded in PLAID can vary geometry and mesh across samples and, within one sample, across time steps, so training data can reflect remeshing and element erosion without bespoke converters.","Because all datasets share the same reader and scoring metric, surrogate models can be compared across structural mechanics and CFD using a single evaluation protocol.","The released datasets provide a common testbed for generalization under geometric variability, with baseline errors that future submissions can update through the open leaderboards.","The data model supports fields that appear and disappear over time, fields defined on different subdomains, and meshes linked across time steps, covering tasks like surface-only outputs on a blade embedded in a fluid domain."],"supporting_citations":[{"why":"Defines the underlying hierarchical standard that the PLAID datamodel is built on for meshes, fields, and zones.","marker":"[65]"},{"why":"A large physics-dataset collection that motivates the need for a more general datamodel; PLAID positions itself as covering the unstructured and remeshed geometries it omits.","marker":"[63]"},{"why":"The existing airfoil RANS dataset that the authors re-encode in PLAID, providing continuity with an established CFD benchmark.","marker":"[55]"},{"why":"Introduces the graph-network architecture that serves as a baseline learning method for field prediction.","marker":"[25]"},{"why":"Introduces the morphing-Gaussian-process method used as a baseline on steady-state, geometrically varying datasets.","marker":"[19]"},{"why":"Defines the Fourier-neural-operator variant used as the baseline on the time-dependent dynamics dataset.","marker":"[86]"},{"why":"Supplies the dataset tooling used for storage, streaming, and sharing of the PLAID collections.","marker":"[66]"}],"fun_headline_variants":["PLAID unifies physics data for ML","Unified schema makes physics data ML-ready","Six open datasets, one framework: PLAID for physics ML","PLAID standardizes heterogeneous simulation data for ML","A common data format for physics-based machine learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes, rather than proves, that this schema can represent every combination of the advertised features—simultaneous remeshing, mixed element topologies, multiple spatial dimensions, and time-varying fields—and its evidence is six datasets that happen to encode successfully.","fun_headline_variants_meta":{"raw":{"variants":["PLAID unifies physics data for ML","Unified schema makes physics data ML-ready","Six open datasets, one framework: PLAID for physics ML","PLAID standardizes heterogeneous simulation data for ML","A common data format for physics-based machine learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001088,"raw_usage":{"total_tokens":4539,"prompt_tokens":928,"completion_tokens":3611,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":3537}},"tokens_in":544,"tokens_out":3611,"duration_ms":31875,"temperature":1.0,"reasoning_tokens":3537,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:38:09.922607+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Attempt to encode a single simulation that simultaneously has non-conforming mixed-dimensional interfaces, topology change between time steps, and fields defined on cell centers that appear and disappear; if the PLAID library or schema requires modification or loses information, the generality claim is overstated.","supporting_citations":[{"cited_title":"Seven keys for practical understanding and use of cgns","cited_arxiv_id":null,"evidence_quote":"Defines the underlying hierarchical standard that the PLAID datamodel is built on for meshes, fields, and zones."},{"cited_title":"Airfrans: High fidelity computational fluid dynamics dataset for approximating reynolds-averaged navier–stokes solutions","cited_arxiv_id":null,"evidence_quote":"The existing airfoil RANS dataset that the authors re-encode in PLAID, providing continuity with an established CFD benchmark."},{"cited_title":"Domain Agnostic Fourier Neural Operators, 2023","cited_arxiv_id":null,"evidence_quote":"Defines the Fourier-neural-operator variant used as the baseline on the time-dependent dynamics dataset."},{"cited_title":"huggingface/datasets: 2.8.0, December 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset tooling used for storage, streaming, and sharing of the PLAID collections."}],"review_version":1}