REVIEW 4 major objections 5 minor 8 references
A data model to connect the ESO Data Processing System (EDPS) to ELT data archives
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that astronomical data-reduction workflows can be derived automatically from a declarative data model, with no separate task definitions or workflow rules, if the workflow engine uses backward chaining.
desk verdict A plausible design proposal for ELT pipelines whose central claim is asserted, not demonstrated, and undermined by its own footnote admitting a manual workflow is still used. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the backward-chaining workflow engine, paired with a declarative data model parsed directly from the METIS Data Reduction Library design document. The LaTeX design document is parsed into Python dataclass instances, which are converted into Python files that generate database tables and views, with FITS headers mapped to database columns. The engine starts from the desired end product and recursively works backwards, constructing associations and jobs in an order that keeps the dependency graph connected. This connected graph is what allows local relationships to substitute for global workflow rules.
What would settle it
Run the backward-chaining SPDE engine on the full METIS pipeline and compare its automatically generated job graph to the manually created workflow the authors provide in footnote 3. If the data model alone does not reproduce the same set of jobs, or a demonstrably valid alternative, on a real non-trivial case, the core proposition fails; alternatively, find a single workflow structure from the forward-chaining literature that a backward-chaining engine still cannot express as local data-model relationships.
Extended reading notes
Core claim
The core proposition is that explicit task definitions and workflow rules are unnecessary: a complete set of jobs can be constructed directly from the data model, provided the model specifies not only the data items but also the relationships between different types of input data items. The paper extends the data-driven approach to the METIS instrument and adapts the ESO Data Processing System to support backward chaining. In a backward-chaining system, the workflow engine starts by interpreting the task that creates the desired end product and recursively works backwards towards the raw data, keeping the dependency graph connected from the outset. This connectedness makes it trivial to propagate information across the graph, such as which calibration file should be used, solving a class of problems that forward-chaining systems had to handle with overarching workflow rules.
Load-bearing premise
The load-bearing premise is that backward chaining resolves every workflow-structure limitation that forced earlier projects to use explicit workflow rules, yet this is illustrated with one example and the authors still supply a manually created workflow (footnote 3).
Editorial extensions
If this is right
- Workflow construction for ELT archives becomes automatic, so the same design document that describes data products would also produce the processing jobs.
- Calibration selection rules, such as which master flat to use, become local properties of the dependency graph rather than separate global workflow rules, reducing duplication and inconsistency.
- EDPS and similar engines can be adapted for backward chaining without sacrificing backward compatibility, since forward and backward chaining typically produce the same job graph.
- Dynamic and lazy job execution become easier, because the engine knows the final goal from the start and can interleave job creation with execution.
Reading between the lines
- The same backward-chaining data-model approach could generalize beyond astronomy to any scientific data-processing pipeline that has a declarative description of data products, though the paper only demonstrates it on METIS.
- If the claim holds, the separation between data model, task definitions, and workflow rules collapses into a single artifact, changing how pipeline software is designed and maintained.
- The paper's own admission that a manually created workflow is still provided suggests the data model's expressiveness is not yet proven; a stronger test would be to express an existing full pipeline entirely in data-model terms.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that for the METIS AIT archive, astronomical pipeline workflows can be generated directly from a declarative data model if the workflow engine uses backward chaining. It describes parsing the METIS Data Reduction Library design document into a machine-readable model, claims that explicit task definitions and workflow rules can be subsumed by the data model and local relationships, introduces SPDE (a fork of ESO's EDPS with backward chaining), and illustrates the claimed advantage with one small example. It closes with plans to compare EDPS and SPDE in the METIS archive.
Significance. If the central claim held, it would simplify pipeline construction and improve traceability in ELT archives, and the proposed EDPS/SPDE integration is timely. The paper's strengths are the clean three-way separation of data model, task definitions, and workflow rules; the concrete illustration in Figure 1; and the explicit plan to compare forward- and backward-chaining engines. However, the manuscript currently offers no machine-checked proofs, reproducible implementation, or empirical results: the parser is only "available on demand" and SPDE is a pre-release to be obtained from the authors, so an independent reader cannot verify the claims. The central assertion is therefore best judged as a design proposal whose main claim remains unproven. The claim is falsifiable in principle, which is a strength, but the paper does not yet perform the falsification test.
major comments (4)
- [Section 4, "Our core proposition" and footnote 3] The paper's central claim that explicit workflow rules can be eliminated because they are subsumed in the data model is directly qualified by footnote 3, which states that a manually created workflow is still provided for the METIS pipeline. If backward chaining truly makes workflow rules unnecessary, then a manually created workflow should not be needed. As written, the manuscript both asserts the strong proposition and concedes that it is not realized in practice. The authors should either provide a concrete demonstration that the manual workflow is redundant or restrict the claim to a future goal.
- [Sections 4 and 5, Freudling et al. 2013 and Figure 1] The argument that the limitations of earlier data-model-driven workflow generation were caused by the workflow engine, and that backward chaining resolves them, is not substantiated. The paper does not enumerate the specific limitations identified by Freudling et al. 2013, nor does it show that backward chaining addresses each one. The single example in Figure 1 concerns a propagation choice between two associations; it does not demonstrate that grouping, ordering constraints, branching, iteration, or external runtime conditions can all be encoded as local data-model relationships. A fuller argument or a set of representative examples covering these workflow structures is needed before the central claim is supported.
- [Section 4, task definitions versus data model] The core proposition states that "there is no need for explicit task definitions and workflow rules, because they can be subsumed in the data model," but the same section defines task definitions as "the logic and processing code for creating data items using other data items as input." Processing code cannot literally be subsumed by a data model. This is either an inconsistency or an ambiguous use of the term "task definitions." The authors should clarify what precisely is eliminated: if only workflow rules and task-association declarations are eliminated, while task code remains separate, then the proposition should be reworded to say so.
- [Section 6 and Section 7, SPDE evidence] The paper's practical claim depends on SPDE, a fork of EDPS with backward-chaining support, but no code, tests, or equivalence results are shown. Section 6 says only that the authors should be contacted for access to the pre-release, and Section 7 states that the comparison between EDPS and SPDE "will be performed" in the future. Since the reader cannot inspect or run SPDE, the central claim is not verifiable from the manuscript. At minimum, the paper should include a minimal working demonstration, a publicly accessible repository, or an explicit statement that the contribution is a design proposal without implementation validation.
minor comments (5)
- [Section 3] The sentence "Software for this conversion are available on demand" should be replaced by a persistent repository identifier or DOI so that the parser can be inspected and reused by other groups.
- [Figure 1] The caption is dense and difficult to follow; a legend explaining the meaning of solid, dashed, crossed-out, and orange dashed associations, as well as the numbered job creation order, would greatly improve readability.
- [Section 1] The phrase "XML variants for Euclid, MUSE, and MICADO" is vague; the authors should specify which XML-based data models or schemas are meant and cite the relevant documentation.
- [Section 6] The distinction between EDPS and SPDE is described only as "backward chaining support"; a sentence describing the fork's architecture and whether any modifications were made beyond the chaining strategy would help the reader assess the claim.
- [Section 5] The statement that propagating information over the edges of the graph is "trivial" is too strong; while backward chaining makes the dependency graph connected, implementing pull-based propagation in a real workflow engine remains a nontrivial engineering task. A more measured wording would be more convincing.
Circularity Check
No circular derivation: the central proposal is an unsupported design claim, but it does not reduce to its inputs or to a self-citation chain.
full rationale
The paper does not derive any quantitative result from fitted parameters or from equations. Its central claim (Section 4) is that explicit task definitions and workflow rules can be subsumed in a data model when a backward-chaining engine is used. This claim is argued with one illustrative example (Figure 1) and is explicitly acknowledged to be not yet realized in practice, since footnote 3 states that a manually created workflow is still provided for METIS. The references to prior work by the same authors (Buddelmeijer et al. 2020 for MICADO; the SPDE fork developed by the authors) provide context and tooling but are not used as proof of the core proposition. The limitations cited from Freudling et al. 2013 are external, and the paper's argument that backward chaining resolves them is an assertion, not a derivation from a self-referential definition. No equation or claim in the paper reduces, by construction, to its own inputs. The self-citation to Buddelmeijer et al. 2020 is minor and not load-bearing; the central claim stands (or falls) on the strength of the proposed backward-chaining mechanism, which is untested but not circular. The paper's own admission of a manually created workflow is a limitation, not circularity. Therefore the appropriate circularity score is low, reflecting only the presence of a minor self-citation without circular force.
Assumptions & free parameters
assumptions (3)
- domain assumption The METIS Data Reduction Library Design document is complete and internally consistent enough that parsing it yields a valid data model.
- domain assumption A backward-chaining workflow engine resolves the limitations of forward-chaining engines for deriving workflows from a data model.
- domain assumption All workflow relationships can be encoded as local relationships between data item types.
invented entities (1)
-
SPDE (sdpe)
Cite this review
Pith. "Pith review of A data model to connect the ESO Data Processing System (EDPS) to ELT data archives." pith.science (2026). https://pith.science/paper/3ZBVT3E6
@misc{pith2026250600899,
author = {Pith},
title = {Pith review of: A data model to connect the ESO Data Processing System (EDPS) to ELT data archives},
year = {2026},
howpublished = {\url{https://pith.science/paper/3ZBVT3E6}},
note = {Machine review of arXiv:2506.00899}
}
read the original abstract
We extend the data-driven approach for astronomical pipelines to the METIS instrument and integrate the ESO Data Processing System (EDPS) by adapting it to support backward chaining.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archive author booktitle chapter collaboration edition editor eid eprint howpublished institution journal key month note number numpages organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.con...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....
- [3]
-
[4]
METIS: The Mid-infrared ELT Imager and Spectrograph
Brandl , B., Bettonvil , F., van Boekel , R., Glauser , A., Quanz , S., Absil , O., Amorim , A., Feldt , M., Glasse , A., G \"u del , M., Ho , P., Labadie , L., Meyer , M., Pantin , E., van Winckel , H., & METIS Consortium 2021, The Messenger, 182, 22. 2103.11208
work page Pith review arXiv 2021
-
[5]
Data model as agile basis for evolving calibration software
Buddelmeijer , H., Verdoes Kleijn , G. A., & Leschinski , K. 2020, in Astronomical Data Analysis Software and Systems XXIX, edited by R. Pizzo , E. R. Deul , J. D. Mol , J. de Plaa , & H. Verkouter , vol. 527 of Astronomical Society of the Pacific Conference Series, 127. 2011.04516
work page Pith review arXiv 2020
-
[6]
M., Ballester , P., Forchi , V., Garc \' a-Dabl \'o , C
Freudling , W., Romaniello , M., Bramich , D. M., Ballester , P., Forchi , V., Garc \' a-Dabl \'o , C. E., Moehler , S., & Neeser , M. J. 2013, , 559, A96. 1311.5411
arXiv 2013
-
[7]
Adaptive Data Reduction Workflows for Astronomy -- The ESO Data Processing System (EDPS)
Freudling , W., Zampieri , S., Coccato , L., Podgorski , S., Romaniello , M., Modigliani , A., & Pritchard , J. 2024, , 681, A93. 2311.03822
work page Pith review arXiv 2024
-
[8]
Mwebaze , J., Boxhoorn , D., & Valentijn , E. A. 2013, Experimental Astronomy, 35, 131
work page 2013
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.