{"id":"dadfbf82-0001-4e67-813f-a79259c31be2","arxiv_id":"2507.22669","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"SOAP is a new Python package for calculating halo and galaxy properties from cosmological simulations, supporting parallel processing, multiple halo finders, and integration with the SWIFT framework.","lead":"This paper presents SOAP, a Python package that computes hundreds of physical properties for galaxies and dark matter halos in cosmological simulations. The package runs in parallel, works with several halo finders, and has already produced catalogs for the FLAMINGO simulation suite.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper provides no repository link, version, tests, or algorithm specification, so the central claims of consistent multi-finder property calculation and parallel efficiency cannot be independently verified.","rationale":"The reader identified the weakest assumption as the correctness of the SOAP implementation and the lack of independent checks. My stress-test pass confirms this is the most load-bearing concern: the paper's scientific claim is that SOAP provides consistent property calculation across multiple halo finders, and that consistency is a computational property that cannot be assessed from prose alone. I looked for an alternative, sharper internal gap in the algorithm description—such as the unspecified choice of halo center for SO radii and the interpolation scheme for the expanding-sphere search—and although these are concrete ambiguities, they are instances of the same underlying issue: the implementation is not verifiable. The package's documented use in FLAMINGO and in more than 20 papers is real evidence of adoption, but it does not replace unit tests, analytic checks, or a code link; catalogues could be widely used yet still contain systematic biases in edge cases. Therefore the conditional verdict is appropriate: accept only after the authors provide access to the code, a versioned release, and a validation demonstration. My proposed concrete test specifically targets the claimed cross-finder consistency by comparing outputs when the same particle set is routed through different converters, which isolates the property-calculation layer from the halo-finder differences.","tokens_in":4640,"tokens_out":5513,"duration_ms":69650,"concrete_test":"Make the source repository and a release version (with DOI) publicly available and add a validation test: construct a synthetic halo with a known analytic density profile (e.g., a Hernquist sphere) at finite resolution, generate inputs in the format of each of the four supported halo finders, and run SOAP on all four. Verify that computed SO masses, radii, and aperture properties agree with the analytic values within a stated tolerance, and that identical particle sets supplied through different converters produce identical outputs. Publish the benchmark results and test artifacts in the repository.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that SOAP computes over 250 halo and galaxy properties consistently across four halo finders, thereby isolating genuine structure-finder differences. For this to hold, the core algorithms (spherical-overdensity radii, aperture masses, bound-particle properties) and the four halo-finder converters must be correct and mutually comparable. The manuscript supplies no source-code link, version, test suite, analytic validation, or algorithm details. In particular, the SO radius determination is only described as 'constructing expanding spheres until the target density limit is reached'; it does not state which center is adopted, how the discretely sampled density is interpolated, or how periodic boundaries are handled. Since every subsequent property depends on these choices, an undetected bug or ambiguous convention would bias all catalogues. The FLAMINGO adoption and the 20+ publications demonstrate usage, not correctness. This is the weakest load-bearing link in the paper's argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces SOAP, a Python package for computing halo and galaxy properties from cosmological simulations post-processed with a subhalo finder. SOAP is designed to accept input from multiple halo finders, compute over 250 properties under several halo definitions, run in parallel via mpi4py, and output HDF5 catalogues compatible with swiftsimio. The authors report that SOAP has already been used to produce catalogues for the FLAMINGO simulations and that these catalogues have supported more than 20 publications. The paper consists of a summary, a statement of need, and a bulleted overview of features; it does not present validation tests, benchmarks, algorithmic details, a code repository link, or a version number.","tokens_in":4790,"tokens_out":2276,"duration_ms":30623,"significance":"If the implementation is correct and the claims are supported, SOAP would fill a genuine need: a community-shared, multi-finder, parallel tool for consistent halo/galaxy property calculation. The design choices described—lazy property evaluation, per-property filters, automatic PDF documentation, and integration with swiftsimio—are sensible and potentially useful. The strongest parts of the submission are the clear identification of the multi-finder comparison problem and the emphasis on separating genuine structure-finder differences from property-definition differences. However, the paper currently provides no independent evidence of correctness: no source-code link, no tests, no analytic validation, and no performance measurements. The FLAMINGO adoption claim demonstrates usage but not correctness. The central methodological contribution—consistent property calculation across HBT-HERONS, SubFind, VELOCIraptor, and ROCKSTAR—remains unverifiable from the manuscript as submitted.","major_comments":[{"comment":"The paper claims that SOAP \"scales efficiently across multiple compute nodes\" and that parallel processing is \"a unique feature,\" but it presents no parallel performance measurements, no scaling study, and no comparison with the cited packages. Since parallel efficiency is one of the two headline practical advantages stated in the abstract, this is a load-bearing claim. Provide at least weak- and strong-scaling tests on a representative snapshot, or explicitly temper the claim to \"supports parallel processing.\"","section":"Statement of Need, paragraph 2"},{"comment":"The central claim of consistency across halo finders rests on the SO radius algorithm, which is described only as \"constructing expanding spheres until the target density limit is reached.\" The manuscript does not specify which center is used for the expansion, how the density is estimated from a discrete particle distribution, how periodic boundary conditions are handled, how convergence is decided, or how the result differs between hydrodynamics and dark-matter-only runs. These details are essential because all subsequently computed properties depend on the SO radius. Please specify the algorithm precisely or point to a specific versioned code location where the implementation can be inspected.","section":"Statement of Need, paragraph 3 and Overview of Features"},{"comment":"No source-code repository, release version, or DOI is provided, and no test suite or validation results are presented. For a software paper, this means the stated features—four halo-finder converters, the lazy property system, and the 250+ property set—cannot be independently checked. Include a repository link and version number, and add at least basic validation (e.g., unit tests, comparisons with analytic cases such as an isothermal sphere or a Hernquist profile, and cross-checks against catalogues from existing codes) before the claims of consistent multi-finder property calculation can be assessed.","section":"Whole manuscript"}],"minor_comments":[{"comment":"There are several spacing typos: \"IntroductionSOAP\" in the Summary and \"aPython\" in the Abstract should be \"Introduction SOAP\" and \"a Python.\"","section":"Summary and Abstract"},{"comment":"The phrase \"units that areh-free\" is missing a space and should read \"units that are h-free.\"","section":"Overview of Features, last bullet"},{"comment":"The CAESAR reference (Narayanan et al., 2023) is formatted as a GitHub repository without a version or access date; consider citing the package documentation or a release DOI if one exists.","section":"References"},{"comment":"The claim that SOAP produces catalogues \"used in more than 20 publications\" is supported only by a footnote URL; since the number is time-dependent and not part of the technical content, consider moving it to a less prominent place or removing it.","section":"Statement of Need, paragraph 1"}],"recommendation":"major_revision","confidential_remarks":"For a JOSS-style software paper the absence of a repository link and version is unusual and would likely be caught during editorial screening. The self-citation pattern (FLAMINGO, HBT-HERONS, and the authors' own papers) is not itself a problem, but it should not be used as a substitute for validation. If the authors can add a link, a version, a minimal test suite, and a short validation or benchmark section, the paper could become acceptable. I do not see evidence of a fundamental design flaw that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful thing here is the tool itself: SOAP computes 250+ halo and galaxy properties from SWIFT snapshots post-processed by four different subhalo finders, with lazy evaluation, mpi4py parallelism, subvolume handling, and swiftsimio-compatible output. That is a real contribution. The paper describes the architecture clearly, and the fact that the package produced FLAMINGO catalogues used in 20+ papers is strong evidence of adoption and, at least in practice, of correctness. I believe the authors when they say the converters work; they have used them extensively. The multi-finder consistency story is also the right framing: separating property calculation from structure finder identification is useful for quantifying finder systematics, and that goal is legitimate.\n\nThe soft spots are exactly what the reader and the stress-test note flag, and I don't think they are manufactured. The manuscript gives no repository URL, no version, no test suite, no analytic validation, and no benchmark numbers. The spherical overdensity description (\"constructing expanding spheres until the target density limit is reached\") leaves out the center convention, the interpolation scheme, and the periodic boundary treatment. Every property downstream depends on those choices, so this is not pedantry. The parallel scaling claim is asserted without evidence. For a software paper, these matter: the code is the artifact, and the paper should point to it and give enough detail for a competent reader to trust the numbers. The lack of a link is the single biggest omission.\n\nI do not think the paper's central concept is flawed. The algorithm choices are standard, the code appears to have been battle-tested, and there is no circularity problem because no physical result is derived from fitted parameters. The self-citations to FLAMINGO are appropriate: that is where the code is used.\n\nWho is this for? Anyone working with SWIFT or EAGLE simulations who needs ready-made halo catalogues, and anyone comparing subhalo finders. That is a real and fairly broad audience.\n\nMy recommendation: this deserves a serious referee, but the paper is not acceptable as-is. The authors need to add a Software Availability section with a DOI or repository link and version, include at least one validation test with an analytic solution or a convergence check, specify the SO algorithm in enough detail, and give at least one strong/weak scaling plot. With those additions, it would be a solid JOSS-style paper. Without them, the scientific claims are not independently checkable, so I would ask for a major revision before accepting.","headline":"A genuinely useful software description that deserves review, but the manuscript as submitted lacks a code link, tests, and validation details, so it needs revision before acceptance.","tokens_in":5339,"tokens_out":1391,"would_cite":true,"duration_ms":20720,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SOAP computes the same 250+ halo and galaxy properties for any supported halo finder, making the finders themselves the only variable in comparisons.","keywords":["SOAP","spherical overdensity","halo properties","galaxy properties","subhalo finders","cosmological simulations","parallel computing","HDF5"],"falsifier":"A concrete check would be to run SOAP on an isolated dark-matter halo with a known analytic density profile (for example a Navarro-Frenk-White halo) and compare the spherical-overdensity radii, enclosed masses, and concentrations against exact values; any discrepancy larger than the nominal numerical tolerance would show that the property calculations are not correct. A second, targeted check is to feed SOAP the exact same set of particles through the converters of two different halo finders while keeping the bound-particle lists identical, and confirm that every property that depends only on particle membership is numerically identical; if property values depend on which finder supplied the catalogue, the converters are not actually standardizing the calculations.","tokens_in":4447,"feed_emoji":"🌌","tokens_out":7097,"duration_ms":69696,"temperature":0.7,"pith_summary":"SOAP is a Python package that takes a subhalo catalogue from a cosmological simulation and computes over 250 halo and galaxy properties for every object, under four different halo definitions. The paper's central claim is that because SOAP places property calculation in a single layer independent of the subhalo finder, researchers can compare catalogues produced by different structure-finding algorithms without the confounding differences in how each finder happens to define and compute those properties. The package is designed to run in parallel and on subvolumes, so it can handle the largest current hydrodynamic simulations, and it has already produced catalogues for the FLAMINGO simulation suite that are used in more than twenty publications. If the paper is right, SOAP gives the community a shared, flexible tool that turns halo-finder comparisons into controlled experiments rather than apples-to-oranges mismatches.","feed_headline":"SOAP computes 250+ galaxy properties from any halo finder","feed_subtitle":"It gives cosmologists a single property layer, so differences in catalogues come from the finders themselves.","key_machinery":"The load-bearing mechanism is the converter layer plus the standard property-computation core. Each supported halo finder contributes only a script that translates its subhalo catalogue into SOAP's standard format; thereafter, every property is computed by the same code paths regardless of which finder identified the objects. Spherical-overdensity radii are computed by growing spheres until the target density is reached, and all particles within the radius, bound or unbound, are used for SO properties. Properties are defined lazily and can be filtered by user criteria, so only requested quantities are computed, which together with mpi4py-based parallelism and independent subvolume processing is what makes the package efficient on large simulations.","core_discovery":"The central discovery is a working design for separating the identification of bound structures from the calculation of their physical properties. SOAP accepts subhalo catalogues from four widely used finders, converts each into a standard format, and then recomputes properties with identical definitions for every object, including spherical-overdensity radii found by expanding spheres until a target density is reached. With four halo definitions (subhalo particles, spherical overdensity, fixed projected apertures, and fixed physical apertures) and more than 250 lazily computed properties, the output is a single HDF5 catalogue with consistent units, readable by the swiftsimio package. The paper also reports that this design has been exercised on the FLAMINGO simulations, generating catalogues used in over twenty published studies.","pith_inferences":["The paper leaves implicit that the same consistency could be used to quantify finder-induced scatter: by running one simulation's snapshots through several finders and comparing the same SOAP-defined properties, cosmological analyses could attach a systematic uncertainty to halo masses and concentrations.","If the converter layer is as generic as claimed, extending SOAP to a new halo finder would be a small scripting task, potentially making the package a community standard that new finders are expected to support.","The lazy, filterable property system suggests a path toward very large parameter sweeps: because only requested properties are computed, SOAP-based pipelines could produce minimal catalogues for specific observables, reducing storage costs as simulation volumes grow.","A natural next step is to apply SOAP to existing simulations with published halo catalogues beyond FLAMINGO and confirm that recomputed properties reproduce the published values for the same objects, which would also serve as the validation the manuscript does not include."],"forward_implications":["Users can compute 250+ properties across four halo definitions with a single tool, making large simulations analyzable by many researchers without each recomputing their own.","Halo-finder differences can be isolated: inconsistent property definitions no longer contaminate comparisons, so remaining catalogue differences reflect the structure-identification method itself.","The package scales to large simulations via parallel processing and independent subvolume runs, and it handles both dark matter-only and hydrodynamic runs with no parameter changes.","Output HDF5 catalogues integrate with swiftsimio for unit handling and are spatially sorted for fast subvolume loading.","The FLAMINGO catalogues generated with SOAP are already used in more than 20 publications."],"supporting_citations":[{"why":"Demonstrates that different subhalo finders produce significantly different catalogues, establishing the need for a consistent property-calculation layer.","marker":"Forouhar Moreno et al., 2025"},{"why":"Supplies the ROCKSTAR catalogue format that SOAP's converter reads for one of the four supported finders.","marker":"Behroozi et al., 2013"},{"why":"Supplies the SubFind catalogue format and the original bound-mass definitions that SOAP builds on for subhalo properties.","marker":"Springel et al., 2001"},{"why":"Supplies the VELOCIraptor catalogue format and its bound-mass convention used in one of SOAP's converters.","marker":"Elahi et al., 2019"},{"why":"Supplies the HBT+ catalogue format used by HBT-HERONS, one of the finders SOAP supports.","marker":"Han et al., 2018"},{"why":"Provides swiftsimio, the package that SOAP output integrates with for unit handling and visualization.","marker":"Borrow & Borrisov, 2020"},{"why":"Provides the SWIFT simulation snapshots and metadata that SOAP originally reads, including spatial sorting and unit information.","marker":"Schaller et al., 2024"}],"fun_headline_variants":["SOAP computes 250+ galaxy properties from any halo finder","SOAP recalculates galaxy properties identically across finders","One property package for all four halo finders","SOAP: parallel, consistent halo properties for SWIFT","Standardized galaxy properties via SOAP's spherical overdensity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that its implementation is numerically correct and physically consistent, yet it presents no validation against analytic solutions, no unit tests, and no repository link or version identifier, so a reader cannot independently check correctness from the manuscript.","fun_headline_variants_meta":{"raw":{"variants":["SOAP computes 250+ galaxy properties from any halo finder","SOAP recalculates galaxy properties identically across finders","One property package for all four halo finders","SOAP: parallel, consistent halo properties for SWIFT","Standardized galaxy properties via SOAP's spherical overdensity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00029,"raw_usage":{"total_tokens":1651,"prompt_tokens":855,"completion_tokens":796,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":714}},"tokens_in":471,"tokens_out":796,"duration_ms":9929,"temperature":1.0,"reasoning_tokens":714,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:24:06.917063+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check would be to run SOAP on an isolated dark-matter halo with a known analytic density profile (for example a Navarro-Frenk-White halo) and compare the spherical-overdensity radii, enclosed masses, and concentrations against exact values; any discrepancy larger than the nominal numerical tolerance would show that the property calculations are not correct. A second, targeted check is to feed SOAP the exact same set of particles through the converters of two different halo finders while keeping the bound-particle lists identical, and confirm that every property that depends only on particle membership is numerically identical; if property values depend on which finder supplied the catalogue, the converters are not actually standardizing the calculations.","supporting_citations":[],"review_version":1}