{"id":"d475a9e1-cc83-47a2-9c14-54996937a381","arxiv_id":"2412.05021","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"NEAO is a domain ontology that standardizes descriptions of neuroelectrophysiology analysis steps and links them to software implementations, and the paper demonstrates SPARQL queries over annotated provenance from three example analyses.","lead":"The authors built a new ontology, NEAO, that gives standard names and structures to the steps of analyzing brain electrophysiology data, including which software and parameters were used. A generalist might care because this is a concrete attempt to make complex scientific analysis pipelines machine-searchable, so that other researchers can find and reuse exactly how a result was produced.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NEAO's query demonstrations rely on a bespoke Alpaca-to-NEAO mapping and manual code decorators, so the claimed interoperability benefit is only shown for instrumented workflows.","rationale":"The reader identified the mapping between Alpaca/PROV-O and NEAO as the weakest assumption, and my analysis independently converges on the same point: the central claim's utility depends on NEAO annotations being attached to provenance, and the attachment mechanism demonstrated is bespoke to the three example scripts. However, the paper is explicitly framed as an illustration ('we illustrate how using an ontology... can assist'), and the authors transparently state that the association between NEAO and data creation must be performed by the collecting tool and ideally would be automated in the future. Thus the concern is a limitation on external validity rather than an internal contradiction or an overclaim beyond the presented scope. The artifact is real, reproducible, and the queries are internally consistent; the demonstrated proof-of-concept supports the claim that NEAO can enable such queries when annotations are available. The absence of evidence for generalization does not overturn the acceptance of the paper as a domain-ontology engineering contribution with a feasibility demonstration. I therefore keep the reader's ACCEPT verdict unchanged, while noting that the generalizability concern is the key issue to address in future work or a revised manuscript.","tokens_in":35728,"tokens_out":4247,"duration_ms":46809,"concrete_test":"Run a fourth analysis using an unmodified third-party toolbox function (e.g., MNE's spectral computation) with Alpaca provenance capture, then apply Listings 1–3 verbatim and attempt the query pattern from Table 6A. If the queries fail or require new hand-written mapping entries (additional VALUES for package names, new annotation decorators, or new rules in Listing 3), the generalizability of the mapping is unsupported and the claimed interoperability benefit is limited to instrumented workflows; if the queries succeed unchanged, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that NEAO provides a unified vocabulary enabling queries over analysis provenance—requires NEAO annotations to be attached to provenance, and the paper achieves this through two hand-crafted steps: (i) a __ontology__ decorator manually inserted into each function of the three example scripts, and (ii) SPARQL update queries (Listings 1–3) that map Alpaca/PROV-O provenance to NEAO. Listing 2 hard-codes package-name mappings for only neo, elephant, and scipy, while excluding user-defined functions with a FILTER; Listing 3 only maps container outputs when members already carry NEAO Data annotations. There is no evidence that these mappings or the annotation mechanism generalize to unmodified third-party code (e.g., MNE, FieldTrip) or to other provenance capture tools, which is exactly the 'distinct software codes' interoperability scenario motivating the paper. The authors acknowledge that the association 'must be performed by the tool collecting the provenance information,' but provide no concrete mechanism for automated annotation, leaving the demonstrated benefit limited to the authors' instrumented workflows. This does not invalidate the ontology as a proof of concept, but it weakens the stronger reading of the FAIR-interoperability claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Neuroelectrophysiology Analysis Ontology (NEAO), an OWL ontology that provides a unified vocabulary for describing the atomic steps, data entities, parameters, software implementations, and bibliographic references involved in neuroelectrophysiology data analysis. The ontology was designed from a review of existing electrophysiology toolboxes and literature rather than from the example analyses, and it uses taxonomic grouping and Rector normalization to represent both specific methods and semantic categories such as power spectral density analysis or functional connectivity analysis. The authors then demonstrate the ontology on three real-world analysis scenarios: PSD computation with different methods and toolboxes, ISIH computation from spike-train surrogates, and ISIH computation from artificially generated spike trains. The scripts were instrumented with the Alpaca provenance tool, NEAO annotations were added via Python decorators, and SPARQL update queries mapped the PROV-O/Alpaca provenance to NEAO relations. The resulting knowledge graph is queried to recover which analysis steps, methods, software packages, and parameters produced each output file, supporting the paper's claim that NEAO improves findability, interoperability, and reusability of analysis results.","tokens_in":36053,"tokens_out":9212,"duration_ms":93284,"significance":"If accepted, NEAO addresses a genuine gap: existing biomedical and neuroscience ontologies generally lack the specificity needed to connect electrophysiology analysis methods to their software implementations and parameters. The paper's main contribution is a concrete, reproducible proof of concept. The ontology files, analysis scripts, Alpaca provenance files, SPARQL update queries, and raw query-result CSVs are all provided, and the reported query outputs match the ground truth of the constructed analyses. The ontology was built from an independent toolbox and literature survey rather than fitted to the example analyses, which substantially reduces circularity concerns. A notable strength is that the queries recover distinctions that are genuinely hard to obtain from code inspection alone, such as Welch versus multitaper PSD estimation, Elephant versus SciPy implementations, and uniform spike dithering versus trial shifting.","major_comments":[],"minor_comments":[{"comment":"The text states that Analysis 2.1 used a dithering time of 15 ms, but the Methods section ('Use case analyses', Analysis 2) and Table 9C both report 25 ms for the uniform spike dithering condition. This numerical inconsistency in a displayed query result must be corrected and reconciled across the three locations.","section":"Results, 'Which spike train surrogate generation method was used?'"},{"comment":"The demonstrated interoperability pipeline is tied to Alpaca and to the three instrumented scripts: Listing 2 hard-codes package-name mappings for only neo, elephant, and scipy, and Listing 3 maps container outputs only when their members already carry NEAO Data annotations. The discussion already acknowledges that annotation must be performed by the collecting tool, but the paper should add an explicit sentence in the results or discussion clarifying that the software-implementation mapping is a proof-of-concept for the demonstrated tool stack and that generalization to other provenance tools or unmodified third-party code is future work.","section":"Methods, 'Knowledge graph and SPARQL queries' and Listings 2-3"},{"comment":"The statement that NEAO 'cannot be directly integrated into the OBO Foundry' is appropriate, but the subsequent discussion of future alignment is brief. Since alignment to BFO/RO is a common expectation for biomedical ontologies, the authors should briefly mention whether the lack of alignment affects the practical use of NEAO with existing OBO Foundry tools or reasoners, or whether it only affects formal interoperability with OBO-aligned ontologies.","section":"Discussion, limitations paragraph"}],"recommendation":"minor_revision","confidential_remarks":"The paper is a solid domain-ontology contribution with unusually complete reproducibility artifacts. The main limitation, namely that the NEAO-to-Alpaca mapping is hand-written and package-specific, is acknowledged and does not undermine the proof-of-concept. The only substantive error I found is the dither-time inconsistency between the Results text (15 ms) and the Methods/Table 9C (25 ms), which is easily fixed. The paper fits the scope of a quantitative biology / neuroinformatics venue and I see no citation or novelty concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, this is a solid ontology engineering paper. The thing to know: NEAO is the first ontology I've seen that ties neuroelectrophysiology analysis steps to concrete software implementations, parameter settings, and bibliographic references at this granularity. The authors demonstrate it on three realistic analysis scripts and provide everything: scripts, Alpaca provenance, NEAO annotations, SPARQL updates, and query result CSVs. The query outputs match the known ground truth. That is real, reproducible evidence, and it earns the paper a serious look.\n\nWhat's genuinely new: the binding between the semantic layer (analysis step classes) and the provenance graph. The Rector normalization to create inferred grouping classes (e.g., FunctionalConnectivityAnalysis) is a clean solution to the problem of method ambiguity. The use of SKOS annotations for disambiguation is standard but well applied. The competency questions are sensible and the answers line up.\n\nThe main soft spot is the one the paper half-admits: the mapping from Alpaca/PROV-O to NEAO is hand-written via three SPARQL updates. Listing 2 hard-codes package name mappings for neo, elephant, and scipy and filters out user-defined functions. Listing 3 only maps container outputs when members already carry NEAO Data annotations. So the demonstrated interoperability is real but limited to pipelines that are explicitly instrumented with the __ontology__ decorator and where the mapping rules have been written. The paper says the association 'must be performed by the tool collecting the provenance information,' but does not provide an automated mechanism. This is not fatal—the ontology itself is the contribution, not the mapping—but it means the broader FAIR interoperability claim is supported as a proof of concept, not as a measured result.\n\nThe other limitations are minor and stated clearly: no OBO alignment, no OWL restrictions, limited to extracellular data. These are reasonable design choices for a first release.\n\nWho is this for? Neuroinformaticians and tool developers working on provenance, reproducibility, or metadata standards for electrophysiology. A general biologist would not need it. It deserves peer review: the artifact is available, the evidence is reproducible, and the claims are honestly scoped. My recommendation is to engage with it, but expect revision to tighten the generalization claims and ideally to provide at least one example with a different provenance capture tool.","headline":"A solid, reproducible ontology paper for electrophysiology analysis provenance; the mapping from provenance to NEAO is hand-crafted and limits the interoperability claim to instrumented pipelines.","tokens_in":36522,"tokens_out":2213,"would_cite":true,"duration_ms":22017,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces the Neuroelectrophysiology Analysis Ontology (NEAO), a vocabulary for describing the atomic steps of neuroelectrophysiology data analysis, and shows that annotating captured provenance with NEAO classes lets…","keywords":["ontology","neuroelectrophysiology","data analysis provenance","knowledge graphs","SPARQL","FAIR principles","electrophysiology","analysis workflows"],"falsifier":"Run the three example analyses with the provenance captured by a second, non-PROV-O tool, or by a script that passes parameters through a wrapper so that alpaca:hasParameter does not point directly to the annotated parameter, then apply the transformation rules in Listings 1–3 and check whether the knowledge graph still answers the queries in Tables 7–10 without adding new rules. If it does not, the demonstrated interoperability depends on the hand-crafted mapping.","tokens_in":35557,"feed_emoji":"🧠","tokens_out":5323,"duration_ms":65076,"temperature":0.7,"pith_summary":"Describing how neuroelectrophysiology data were analyzed is hard because different algorithms can produce similar measures and different toolboxes can implement the same algorithm under different names. This paper presents the Neuroelectrophysiology Analysis Ontology (NEAO), a vocabulary that standardizes descriptions of analysis steps, their data inputs and outputs, parameters, software implementations, and bibliographic references. The authors show that when provenance captured during the execution of real analysis scripts is annotated with NEAO classes, SPARQL queries over the resulting knowledge graph can recover which methods, software packages, and parameters produced each output file. The demonstrated queries answer questions that would otherwise require reading the code, such as whether a power spectral density estimate came from the Welch or multitaper method, which toolbox was used, and whether the same filtering parameters were applied.","feed_headline":"Ontology turns neural data analysis steps into searchable queries","feed_subtitle":"Standardized labels for methods, parameters, and tools let researchers find and reuse results without reading code.","key_machinery":"The central object is the NEAO ontology model, a set of OWL classes built around AnalysisStep, Data, and AnalysisParameter, with object properties hasInput, hasOutput, and usesParameter to wire steps to their data and controls, and isImplementedIn and isImplementedInPackage to attach software function and package details. Name ambiguity is handled by controlled skos:prefLabel and skos:altLabel annotations and by bibliographic references via BiRO. The grouping of semantically similar methods is carried by two mechanisms: a primary subclass taxonomy (e.g., PowerSpectralDensityAnalysis above the Welch and multitaper step classes) and the Rector normalization technique, which uses restrictions on properties like hasPurpose to infer additional grouping classes such as FunctionalConnectivityAnalysis. The operational bridge from code to semantics is a set of SPARQL update rules that convert provenance triples (prov:used, prov:generated, alpaca:hasParameter, alpaca:usedFunction) into NEAO properties, plus a container-membership rule for outputs that are collections.","core_discovery":"NEAO models an analysis as a sequence of atomic AnalysisStep instances, each linked by hasInput and hasOutput to Data entities and by usesParameter to AnalysisParameter entities, and each tied to a SoftwareImplementation (Function or Program) and SoftwarePackage through isImplementedIn and isImplementedInPackage. Semantic groupings are asserted in a primary taxonomy and extended by inferred groupings built with the Rector normalization technique, so that, for instance, coherence and cross-correlation steps can both be recognized as functional connectivity analyses. The decisive demonstration is a mapping (Table 11 and Listings 1–3) from the PROV-O-based provenance graph emitted by the Alpaca tool into NEAO classes, followed by SPARQL queries over a knowledge graph built from three real analysis scripts. The queries recover, per result file, the full chain of analysis steps, the specific methods, the software packages and versions, and the parameter values used, without any knowledge of the underlying Python functions.","pith_inferences":["If the NEAO-to-provenance mapping were standardized rather than hand-crafted per script, the same query layer could be attached to any Python analysis that captures provenance, turning the approach into a general reproducibility layer.","The hasPurpose grouping mechanism is a template for cross-method comparisons in other domains; for example, grouping diverse statistical tests by inferential purpose would let users query all significance tests without naming each one.","Because NEAO attaches bibliographic references to method classes, a knowledge graph of annotated results could be mined to ask whether parameter choices differ systematically across toolboxes for the same method.","The paper's stated limitation that AnalysisStep has no OWL restrictions on inputs and outputs suggests that toolbox-specific NEAO modules could add such restrictions without breaking cases where a method returns either an array or a plot."],"forward_implications":["Researchers can query a collection of result files by conceptual content, for example all files containing a power spectral density estimate, without enumerating every algorithm that produces such an estimate.","Provenance captured during script execution, once annotated with NEAO classes, yields method-, parameter-, and software-level answers that currently require inspecting the code or trusting a README.","The ability to infer that two analyses used the same method in different toolboxes, or used equivalent parameters, supports comparisons of analysis results across researchers and laboratories.","The modular structure of the ontology permits future toolbox-specific modules that could carry restrictions describing valid inputs, outputs, and parameters for a given tool."],"supporting_citations":[{"why":"Supplies the provenance-capture method (Alpaca) whose PROV-O output is mapped to NEAO semantics.","marker":"[51]"},{"why":"Provides the toolbox comparison from which NEAO's analysis-step vocabulary is derived.","marker":"[15]"},{"why":"Provides the Rector normalization technique used to infer semantic grouping classes.","marker":"[49]"},{"why":"Motivates the FAIR principles that NEAO is claimed to advance.","marker":"[21]"},{"why":"Provides the open dataset used in the three use-case analyses.","marker":"[69]"}],"fun_headline_variants":["NEAO makes neural analysis steps queryable","Standardized ontology for searchable analysis workflows","Ontology turns neural data steps into reusable queries","NEAO enables findable, reusable analysis provenance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The query demonstrations rely on a hand-written mapping from the provenance graph produced by one specific Python tool (Alpaca) to NEAO semantics; if that mapping cannot be generalized to other provenance capture tools or code structures without bespoke rules, the interoperability benefit is limited to instrumented workflows.","fun_headline_variants_meta":{"raw":{"variants":["NEAO makes neural analysis steps queryable","Standardized ontology for searchable analysis workflows","Ontology turns neural data steps into reusable queries","NEAO enables findable, reusable analysis provenance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1439,"prompt_tokens":975,"completion_tokens":464,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":405}},"tokens_in":591,"tokens_out":464,"duration_ms":6104,"temperature":1.0,"reasoning_tokens":405,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:58:00.453755+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the three example analyses with the provenance captured by a second, non-PROV-O tool, or by a script that passes parameters through a wrapper so that alpaca:hasParameter does not point directly to the annotated parameter, then apply the transformation rules in Listings 1–3 and check whether the knowledge graph still answers the queries in Tables 7–10 without adding new rules. If it does not, the demonstrated interoperability depends on the hand-crafted mapping.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Rector normalization technique used to infer semantic grouping classes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the open dataset used in the three use-case analyses."},{"cited_title":"A., Ulianych, D., Grün, S., Decker, S","cited_arxiv_id":null,"evidence_quote":"Supplies the provenance-capture method (Alpaca) whose PROV-O output is mapped to NEAO semantics."}],"review_version":1}