Pith. sign in

REVIEW 1 major objections 8 minor 35 references

Executable verification protocol matches expert astronomers 95.5%

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 15:43 UTC pith:QFDOAA2Y

load-bearing objection FORMA formalizes expert spectroscopic verification as an executable, auditable protocol — promising concept, but headline numbers rest on a single pipeline run with limited stability evidence. the 1 major comments →

arxiv 2607.06128 v1 pith:QFDOAA2Y submitted 2026-07-07 astro-ph.CO astro-ph.IM

Executable verification through formalized expert reasoning in astronomical spectroscopy

Haosong Wang , Ting Tan , Ji Yao , Jiajun Zhang , Qian Zheng , Christophe Yeche , Jean-Paul Kneib , Huanyuan Shan This is my paper
classification astro-ph.CO astro-ph.IM PACS 95.75.Mn95.85.Kr
keywords verificationscientificauditableconsistencycredibilityevidenceexecutableexpert
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces FORMA (Formalized Observational Reasoning with Auditable Decisions), a computational protocol that reconstructs the reasoning steps a human expert uses when verifying astronomical spectra — extracting evidence, generating physically constrained hypotheses, testing alternatives, and checking consistency — into an executable, auditable workflow. The central claim is that scientific verification, not just prediction, can be formalized as a machine-executable process. Applied to the DESI spectroscopic survey's expert-review catalogue, FORMA operates without dataset-specific training: it takes candidate redshifts from a template-fitting engine as hypotheses to be tested, not as accepted answers, and assigns a credibility score based on whether the physical evidence suffices to support the proposed interpretation. At a medium-or-higher credibility threshold, FORMA produces 331 definite predictions with 95.5% binary agreement with expert-adjudicated classifications. The credibility score correlates with redshift accuracy but is not a proxy for raw signal-to-noise ratio — low credibility reflects insufficient or ambiguous evidence as judged by the verification pipeline, not simply poor data quality. Counterfactual tests show that when diagnostic spectral evidence is removed or made physically inconsistent, FORMA downgrades or rejects the interpretation rather than rubber-stamping a template match.

Core claim

The paper demonstrates that the expert reasoning process underlying astronomical spectral verification — hypothesis generation under physical constraints, systematic testing of alternatives, and adversarial consistency checking — can be decomposed into discrete, executable computational steps whose outputs are auditable. The FORMA credibility score, assigned by an adversarial review agent without a hand-crafted scoring specification, tracks evidential sufficiency rather than data quality: it separates cases where the physical evidence genuinely supports a classification from cases where a template fit merely looks plausible. This is shown through three lines of evidence: (1) 95.5% agreement

What carries the argument

The FORMA protocol decomposes expert spectroscopic verification into four modules and six steps. The Visual Interpreter (Step 1) converts raw spectral data into computational features using wavelet-based detection and Chebyshev continuum fitting, and generates redshift hypotheses via a template-fitting engine (Redrock). The Hypothesis Analyst (Steps 2 and 4) assigns each candidate redshift to an independent LLM agent that evaluates predicted spectral lines against observed features using deterministic tools for Gaussian fitting and wavelength-to-redshift conversion, then synthesises a verdict across all hypotheses. The Analysis Auditor (Steps 3 and 5) performs two-stage adversarial review: a

Load-bearing premise

The credibility rating (HIGH/MEDIUM/LOW) is assigned entirely by an LLM agent using a knowledge base and spectral inspection, without a hand-crafted scoring specification. If the agent's internal calibration of what constitutes sufficient evidence is unstable or sensitive to prompt format, the credibility-gated triage mechanism could route cases incorrectly.

What would settle it

Run FORMA on a large independent sample of expert-adjudicated spectra where the ground truth is known with high confidence, and check whether the credibility score reliably separates correct from incorrect classifications. If the score fails to stratify accuracy beyond what signal-to-noise alone predicts, or if counterfactual evidence ablations fail to trigger downgrades in a substantial fraction of runs, the claim that the protocol tracks evidential sufficiency rather than data quality would be undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the verification-protocol approach generalises, next-generation surveys (Rubin, SKAO, Euclid, 4MOST, DESI II) could deploy automated verification layers that triage which candidate classifications are safe for scientific use before human review, reducing the expert bottleneck at scale.
  • The credibility-score mechanism — which gates acceptance on evidential sufficiency rather than model confidence — could become a standard output of survey pipelines, replacing or complementing binary quality flags that conflate data quality with classification reliability.
  • The counterfactual evidence-ablation methodology provides a template for testing whether any automated reasoning system bases its decisions on physically meaningful evidence rather than pattern matching, applicable beyond astronomy.
  • Accumulated verification records with full evidential provenance could serve as versioned knowledge repositories that preserve operational expertise across long-lived survey collaborations, transferring judgment that is otherwise tacit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 8 minor

Summary. The manuscript introduces FORMA (Formalized Observational Reasoning with Auditable Decisions), a multi-agent LLM-based protocol that formalizes expert spectroscopic verification as an executable workflow. FORMA decomposes visual inspection into six steps: deterministic feature extraction (CWT-based), parallel single-hypothesis evaluation by LLM agents, adversarial feature auditing, hypothesis synthesis, an independent Result Auditor that assigns a credibility rating, and structured report generation. Applied to 1,149 DESI EDR visual-inspection spectra in a zero-shot setting, FORMA produces 331 definite predictions at medium-or-higher credibility with 95.5% binary agreement with expert-adjudicated classes. The credibility score is shown to correlate with redshift residual concentration and to be not merely a proxy for SNR. Counterfactual evidence-ablation tests (100 runs on 2 cases) demonstrate that the Result Auditor rejects physically inconsistent interpretations. The paper also provides a thoughtful critique (Appendix A) of the DESI VI QUALITY flag's limitations in the low-SNR ELG regime.

Significance. The paper addresses a genuine and timely problem: scaling scientific verification for next-generation surveys. Its main conceptual contribution — framing verification as an executable, auditable protocol distinct from prediction or post-hoc interpretability — is well-motivated. Strengths include: (1) a clear architectural separation between deterministic numerical computation and LLM reasoning, which is a sound design principle; (2) the counterfactual ablation methodology (Appendix B) providing falsifiable evidence that the Result Auditor responds to physical evidence rather than pattern-matching; (3) the critique of VI QUALITY in Appendix A, which identifies a real limitation of the DESI inspection protocol; and (4) public code availability. The central claim that expert verification can be partially automated is supported by the DESI evaluation. However, the reproducibility of the headline quantitative result across the full LLM-based pipeline is not established, which is the primary concern detailed below.

major comments (1)
  1. The headline quantitative result — 331 definite predictions with 95.5% binary agreement — derives from a single execution of the full 6-step FORMA pipeline on 1,149 spectra. The only reproducibility evidence is 100 independent re-runs of the Result Auditor (Step 5) on 2 counterfactual spectra (Table B1, Appendix B). However, Steps 2–4 (Single-Hypothesis Evaluation, Feature Auditor, Hypothesis Synthesis) are also LLM-based and involve open-ended reasoning: Step 2 agents assign line statuses (LIKELY/MARGINAL/NOTFOUND/MASKED) and select anchor lines; Step 3 makes KEEP/REMOVE/FLAG judgments; Step 4 selects the best hypothesis and assigns intermediate confidence. These upstream judgments directly determine the line catalogues and hypothesis reports that the Result Auditor receives. If Steps 2–4 exhibit run-to-run variability — which is expected from LLM sampling — the set of spectra receiving
minor comments (8)
  1. The paper uses 'DeepSeek-V4-Pro [20]' as the base LLM but reference [20] is titled 'DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence' with no 'Pro' designation. Please reconcile the model name and cite it accurately.
  2. Fig. 2a: the x-axis label is unclear. The text refers to 'low-or-higher' and 'medium-or-higher' thresholds, but the figure axis should explicitly state what quantity is being thresholded (e.g., credibility tier rank).
  3. Section 'Dataset Composition': the stratification ratio is stated as 4:4:2 (high/medium/low) but it is unclear whether this refers to SNR tiers or VI QUALITY tiers. Please clarify.
  4. The paper states that Redrock returns 'top-N fits' (Step 1 description) but does not specify the value of N used in the evaluation. This should be stated for reproducibility.
  5. Appendix A provides a detailed and valuable critique of VI QUALITY limitations, but this analysis is confined to the appendix. Given that it motivates the entire credibility-score design, a brief pointer in the main text (Section 2) would help readers understand why a new scoring mechanism is needed.
  6. The paper mentions 'a recent systematic evaluation of astronomical VLMs [21]' in the Discussion. Reference [21] (arXiv:2604.24589) appears to be a 2026 preprint; please verify the reference details.
  7. Fig. 3b: the AGN example shows line labels with redshift-shifted wavelengths (e.g., [Ne V] 4242.4 Å) but the rest-frame wavelength of [Ne V] is 3426 Å. At z=0.238, the observed wavelength would be ~4240 Å, which is consistent, but the label format could be clearer about whether values are observed or rest-frame.
  8. The term 'binary agreement' (Section 2) is used for a four-class problem (QSO/LRG/ELG/BGS). It would be clearer to state that this is overall accuracy across all classes, or clarify if a binary QSO-vs-galaxy split was used in the computation.

Simulated Author's Rebuttal

1 responses · 0 unresolved

The referee raises a legitimate and important concern about the reproducibility of the headline quantitative result across the full LLM-based pipeline. We acknowledge this gap and will address it with additional full-pipeline re-runs in the revised manuscript.

read point-by-point responses

  1. Referee: The headline quantitative result — 331 definite predictions with 95.5% binary agreement — derives from a single execution of the full 6-step FORMA pipeline on 1,149 spectra. The only reproducibility evidence is 100 independent re-runs of the Result Auditor (Step 5) on 2 counterfactual spectra (Table B1, Appendix B). However, Steps 2–4 (Single-Hypothesis Evaluation, Feature Auditor, Hypothesis Synthesis) are also LLM-based and involve open-ended reasoning... If Steps 2–4 exhibit run-to-run variability — which is expected from LLM sampling — the set of spectra receiving definite predictions and the resulting agreement rate could change across runs.

    Authors: The referee is correct. The headline 95.5% agreement figure derives from a single full-pipeline execution, and the reproducibility evidence in Appendix B covers only Step 5 (Result Auditor) on two counterfactual spectra. Steps 2–4 are LLM-based and involve open-ended reasoning — line status assignment, anchor line selection, KEEP/REMOVE/FLAG judgments, and hypothesis synthesis — so run-to-run variability in these upstream steps could, in principle, change which spectra receive medium-or-higher credibility and thus alter the agreement rate. We acknowledge this as a genuine gap in the current manuscript. We note that several design choices are intended to reduce upstream variability: Step 1 (Visual Interpreter) is fully deterministic (CWT feature detection, continuum fitting, Redrock hypothesis generation); all wavelength-to-redshift conversions and numerical computations are handled by deterministic tools rather than LLM arithmetic; and the structured prompt protocol constrains each Step 2 agent to evaluate a single hypothesis against pre-computed evidence rather than conducting open-ended exploration. However, these mitigations do not substitute for empirical evidence of stability across repeated runs. We will address this by running the complete FORMA pipeline (Steps 1–6) independently N≥3 times on the full 1,149-spectrum evaluation set and reporting: (i) the variance in the number of definite predictions at the medium-or-higher credibility threshold, (ii) the variance in the binary agreement rate, and (iii) the spectrum-level stability of credibility assignments (the fraction of spectra that change credibility tier across runs). We will add these results as a new supplementary table and discuss them in the main text. If the variance is non-negligible, we will report a revision: no

Circularity Check

0 steps flagged

No significant circularity: FORMA's predictions are evaluated against an external benchmark (DESI VI catalogue) not used during inference; credibility score is self-assigned but validated against external metrics.

full rationale

The paper's central quantitative claim — 331 definite predictions with 95.5% binary agreement — is evaluated against the DESI EDR visual inspection catalogue, which is an external benchmark explicitly not supplied to FORMA during inference ('external pipeline classifications and human visual-inspection labels are used only for evaluation and are not supplied as accepted answers during inference'). The credibility score is self-assigned by the LLM agent ('The credibility rating is assigned entirely by the LLM agent, without a detailed, hand-crafted scoring specification'), which introduces a minor self-referential element: the same system that assigns credibility is also the system whose credibility is being assessed. However, the paper validates this score against independent external metrics (Fig. 2c: redshift residuals concentrate at higher credibility; Fig. 2d: low credibility is not simply a proxy for low SNR), and the counterfactual evidence-ablation tests (Appendix B, 100 independent runs) provide non-circular evidence that the Result Auditor responds to physical evidence rather than rubber-stamping. The feature score is compared against VI QUALITY and shown to fail (Appendix A), which is an honest non-circular finding. No step in the derivation chain reduces to its own inputs by construction. The self-assigned credibility score is a minor weakness but not circular in the strict sense, since it is validated against external data rather than being defined in terms of the validation metric.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 1 invented entities

The ledger captures the operational parameters (thresholds, windows), domain assumptions about expert reasoning and LLM capabilities, and the new credibility metric introduced by the paper.

free parameters (3)
  • Credibility threshold (medium-or-higher) = medium
    Chosen operationally to balance automation fraction and accuracy (Fig 2a).
  • Dataset stratification ratio = 4:4:2
    Chosen for sampling high/medium/low SNR tiers.
  • Redshift verification window = z±0.1
    Set for hypothesis evaluation in Step 2.
axioms (3)
  • domain assumption Expert visual inspection follows a reproducible workflow that can be decomposed into perception, hypothesis inference, and result validation.
    Foundational to the design of FORMA (Method: Design rationale).
  • domain assumption LLMs can reliably apply domain knowledge from a markdown knowledge base to perform physical reasoning and audit tasks.
    Underpins the use of LLM agents for the Hypothesis Analyst and Analysis Auditor modules.
  • ad hoc to paper Numerical computation must be strictly separated from LLM reasoning to ensure reproducibility.
    Architectural choice to isolate deterministic tools from LLM reasoning (Method: Architecture overview).
invented entities (1)
  • FORMA credibility score independent evidence
    purpose: To quantify the sufficiency of supporting evidence as reconstructed by the agent workflow.
    Evaluated against expert classifications and redshift residuals (Fig 2b, 2c) and tested via counterfactual ablation (Fig 3a).

pith-pipeline@v1.1.0-glm · 35302 in / 2169 out tokens · 281318 ms · 2026-07-08T15:43:51.332806+00:00 · methodology

0 comments
read the original abstract

Artificial intelligence has reshaped scientific prediction, but scientific verification remains a human bottleneck. Automated systems can map observations to labels, parameters or hypotheses, yet scientific conclusions require evidence, must satisfy physical consistency, and need explicit testing of alternatives before a decision is made. Here we introduce FORMA (Formalized Observational Reasoning with Auditable Decisions), an executable verification protocol that reconstructs expert reasoning into a workflow: it extracts evidence, generates hypotheses under physical constraints, tests alternatives, and performs auditable consistency checks. Unlike prediction or post-hoc interpretability, executable verification records and tests the evidential path leading to a decision. Astronomical spectroscopy provides a natural testbed, because ambiguous survey spectra are still adjudicated by expert visual inspection. Applied to the Dark Energy Spectroscopic Instrument (DESI) visual inspection catalogue, FORMA combines template-fitting candidate redshifts, spectral evidence extraction and physical audit into an auditable credibility score. A medium-or-higher credibility threshold identifies $331$ definite predictions with $95.5\%$ binary agreement with expert-adjudicated classes, while increasing credibility is associated with improved redshift consistency and higher classification reliability. These results show that automated inference can be coupled to explicit verification, allowing candidate outputs to be evaluated before they enter scientific use.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages · 10 internal anchors

  1. [1]

    Nature Machine Intelligence1, 206–215 (2019) https://doi.org/10.1038/s42256-019-0048-x

    Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence1, 206–215 (2019) https://doi.org/10.1038/s42256-019-0048-x

    work page doi:10.1038/s42256-019-0048-x 2019

  2. [2]

    QuasarNET: Human-level spectral classification and redshifting with Deep Neural Networks

    Busca, N.G., Balland, C.: Quasarnet: Human-level spectral classification and redshifting with deep neural networks. arXiv e-prints (2018) 1808.09955

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  3. [3]

    GitHub repository

    Bailey, S.J., et al.: Redrock: Redshift fitting for spectroperfectionism. GitHub repository. in preparation, https://github.com/desihub/redrock/ (2024)

    work page 2024

  4. [4]

    Teaching LLMs to Speak Spectroscopy

    Ramachandra, N., Ting, Y.-S., Sun, Z., Wells, A., Habib, S.: Teaching LLMs to Speak Spectroscopy (2025). https://arxiv.org/abs/2508.10075

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    https://arxiv.org/abs/2511.08970

    Shao, M., Wang, H., Li, Y., Lin, J., Liu, J., Tan, B., Guo, J., Zhang, Y., Huang, J., Su, J., Sun, Y., Xu, H., Chen, J., Liu, S., Deng, Y., Tong, L., Bai, Y., Wang, C., Ji, K., Zhou, Y.: JW-Flare: Accurate Solar Flare Forecasting Method Based on Multimodal Large Language Models (2025). https://arxiv.org/abs/2511.08970

    work page arXiv 2025

  6. [6]

    Communications Engineering4(1), 184 (2025) https://doi.org/10.1038/s44172-025-00520-4

    Wang, C., Zhang, Y., Li, Y., Hu, X., Mao, Y., Chen, X., Du, P., Wang, R., Wu, Y., Yang, H.,et al.: Star- whisper telescope: an ai framework for automating end-to-end astronomical observations. Communications Engineering4(1), 184 (2025) https://doi.org/10.1038/s44172-025-00520-4

    work page doi:10.1038/s44172-025-00520-4 2025

  7. [7]

    Domain-Specific Agents for Cherenkov Telescope Array Control Software and Gamma-Ray Data Analysis

    Kostunin, D., Jones, E., Sotnikov, V., Sotnikov, V., Golovachev, S., Strube, A.: Enhancing the development of Cherenkov Telescope Array control software with Large Language Models (2025). https://arxiv.org/abs/ 2510.01299

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    https://arxiv.org/ abs/2510.25953

    Tamhane, P.: A Natural Language Interface for Efficient Data Retrieval in SDSS (2025). https://arxiv.org/ abs/2510.25953

    work page arXiv 2025

  9. [9]

    https://arxiv.org/abs/2508

    Wang, H., Zeng, L.: Automated Algorithmic Discovery for Scientific Computing through LLM-Guided Evolutionary Search: A Case Study in Gravitational-Wave Detection (2025). https://arxiv.org/abs/2508. 03661

    work page 2025

  10. [10]

    https: //arxiv.org/abs/2512.19799

    Miao, T., Dai, J., Liu, J., Tan, J., Zhang, M., Jin, W., Du, Y., Jin, T., Pang, X., Liu, Z., Guo, T., Zhang, Z., Huang, Y., Chen, S., Ye, R., Zhang, Y., Zhang, L., Chen, K., Wang, W., E, W., Chen, S.: PhysMaster: Building an Autonomous AI Physicist for Theoretical and Computational Physics Research (2025). https: //arxiv.org/abs/2512.19799

    work page arXiv 2025

  11. [11]

    Radio astronomy in the era of vision-language models: Prompt sensitivity and adaptation,

    Drozdova, M., Lastufka, E., Kinakh, V., Holotyak, T., Schaerer, D., Voloshynovskiy, S.: Radio Astronomy in the Era of Vision-Language Models: Prompt Sensitivity and Adaptation (2025). https://arxiv.org/abs/ 2509.02615

    work page arXiv 2025

  12. [12]

    Egent: An Autonomous Agent for Equivalent Width Measurement

    Ting, Y.-S., Saad, S.M., Liu, F., Shen, Y.: Egent: An Autonomous Agent for Equivalent Width Measurement (2025). https://arxiv.org/abs/2512.01270

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  13. [13]

    Nature Astronomy, 1–10 (2025)

    Stoppa, F., Bulmus, T., Bloemen, S., Smartt, S.J., Groot, P.J., Vreeswijk, P., Smith, K.W.: Textual interpretation of transient image classifications from large language models. Nature Astronomy, 1–10 (2025)

    work page 2025

  14. [14]

    https://arxiv

    Sun, Z., Ting, Y.-S., Liang, Y., Duan, N., Huang, S., Cai, Z.: Mephisto: Self-Improving Large Language Model-Based Agents for Automated Interpretation of Multi-band Galaxy Observations (2025). https://arxiv. org/abs/2510.08354

    work page arXiv 2025

  15. [15]

    https://arxiv.org/abs/2510.17960

    Parker, L., Lanusse, F., Shen, J., Liu, O., Hehir, T., Sarra, L., Meyer, L., Bowles, M., Wagner-Carena, S., Qu, H., Golkar, S., Bietti, A., Bourfoune, H., Casserau, N., Cornette, P., Hirashima, K., Krawezik, G., Ohana, R., Lourie, N., McCabe, M., Morel, R., Mukhopadhyay, P., Pettee, M., Blancard, B.R.-S., Cho, K., Cranmer, M., Ho, S.: AION-1: Omnimodal Fo...

    work page arXiv 2025

  16. [16]

    Feng, X., Wang, Z., Shu, Z., Kneib, J.-P., Torr, P.: LensAgent: A Self Evolving Agent for Autonomous Physical Inference of Sub-galactic Structure (2026) 9

    work page 2026

  17. [17]

    The Astrophysical Journal943, 68 (2023) https://doi.org/10.3847/ 1538-4357/aca5fa

    Lan, T.-W.,et al.: The desi survey validation: Results from visual inspection of bright galaxies, luminous red galaxies, and emission-line galaxies. The Astrophysical Journal943, 68 (2023) https://doi.org/10.3847/ 1538-4357/aca5fa

    work page 2023

  18. [18]

    The Astronomical Journal151, 44 (2016) https://doi.org/10.3847/0004-6256/151/2/44

    Dawson, K.S.,et al.: The sdss-iv extended baryon oscillation spectroscopic survey: Overview and early data. The Astronomical Journal151, 44 (2016) https://doi.org/10.3847/0004-6256/151/2/44

    work page doi:10.3847/0004-6256/151/2/44 2016

  19. [19]

    The Astronomical Journal165(3), 124 (2023) https://doi.org/10.3847/1538-3881/ acacfc

    Alexander, D.M., Davis, T.M., Chaussidon, E., Fawcett, V., Gonzalez-Morales, A.X., Lan, T.-W., Yeche, C., Ahlen, S., Aguilar, J., Armengaud, E.,et al.: The desi survey validation: Results from visual inspection of the quasar survey spectra. The Astronomical Journal165(3), 124 (2023) https://doi.org/10.3847/1538-3881/ acacfc

    work page doi:10.3847/1538-3881/ 2023

  20. [20]

    arXiv preprint arXiv:2606.19348 , year =

    DeepSeek-AI, Xu, A., Lin, B., Xue, B., Wang, B., Xu, B., Wu, B., Zhang, B., Lin, C., Dong, C., Ling, C., Lu, C., Zhao, C., Deng, C., Hou, C., Xu, C., Shao, C., Ruan, C., Sun, C., Dai, D., Guo, D., Yang, D., Chen, D., Li, D., Ji, D., Li, E., Wei, F., Lin, F., Yuan, F., Xia, F., Dai, F., Hao, G., Chen, G., Cao, G., Meng, G., Li, G., Yu, H., Zhang, H., Xu, H...

    work page arXiv 2026

  21. [21]

    A systematic evaluation of vision-language models for observational astronomical reasoning tasks

    Ren, W., Guo, H., Zuo, W., Zhang, X.: A systematic evaluation of vision-language models for observational astronomical reasoning tasks (2026). https://arxiv.org/abs/2604.24589

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  22. [22]

    LSST: from Science Drivers to Reference Design and Anticipated Data Products

    Ivezi´ c,ˇZ., Kahn, S.M., Tyson, J.A.,et al.: LSST: From science drivers to reference design and anticipated data products. The Astrophysical Journal873(2), 111 (2019) https://doi.org/10.3847/1538-4357/ab042c arXiv:0805.2366 [astro-ph]

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.3847/1538-4357/ab042c 2019

  23. [23]

    Publications of the Astronomical Society of Australia37(2020) https://doi.org/10.1017/pasa.2019.51

    Bacon, D.J., Battye, R.A., Bull, P., Camera, S., Ferreira, P.G., Harrison, I., Parkinson, D., Pourtsidou, A., Santos, M.G., Wolz, L., Abdalla, F., Akrami, Y., Alonso, D., Andrianomena, S., Ballardini, M., Bernal, J.L., Bertacca, D., Bengaly, C.A.P., Bonaldi, A., Bonvin, C., Brown, M.L., Chapman, E., Chen, S., Chen, X., Cunnington, S., Davis, T.M., Dickins...

    work page doi:10.1017/pasa.2019.51 2018

  24. [24]

    CMB-S4 Science Book, First Edition

    Abazajian, K.N., Adshead, P., Ahmed, Z., Allen, S.W., Alonso, D., Arnold, K.S., Baccigalupi, C., Bartlett, 10 J.G., Battaglia, N., Benson, B.A., Bischoff, C.A., Borrill, J., Buza, V., Calabrese, E., Caldwell, R., Carlstrom, J.E., Chang, C.L., Crawford, T.M., Cyr-Racine, F.-Y., Bernardis, F.D., Haan, T., Serego Alighieri, S., Dunkley, J., Dvorkin, C., Erra...

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  25. [25]

    Euclid Definition Study Report

    Laureijs, R., Amiaux, J., Arduini, S., Augueres, J.-L., Brinchmann, J., Cole, R., Cropper, M., Dabin, C., Duvet, L., Ealet, A., et al.: Euclid definition study report. arXiv preprint arXiv:1110.3193 (2011)

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  26. [26]

    and Abdurro’uf and Acevedo Barroso, J

    Mellier, Y., Abdurro’uf, Acevedo Barroso, J.A., Ach´ ucarro, A., Adamek, J., Adam, R., Addison, G.E., Aghanim, N., Aguena, M., Ajani, V., Akrami, Y., Al-Bahlawan, A., Alavi, A., Albuquerque, I.S., Alestas, G., Alguero, G., Allaoui, A., Allen, S.W., Allevato, V., Alonso-Tetilla, A.V., Altieri, B., Alvarez-Candal, A., Alvi, S., Amara, A., Amendola, L., Amia...

    work page doi:10.1051/0004-6361/202450810 2025

  27. [27]

    Science China Physics, Mechanics & Astronomy69(3) (2026) https://doi.org/10.1007/s11433-025-2809-0

    Gong, Y., Miao, H., Zhan, H., Li, Z.-Y., Shangguan, J., Li, H., Liu, C., Chen, X., Yuan, H., Zhou, J., Liu, H.-G., Yu, C., Ji, J., Qi, Z., Liu, J., Dai, Z., Wang, X., Zheng, Z., Hao, L., Dou, J., Ao, Y., Lin, Z., Zhang, K., Wang, W., Sun, G., Li, R., Li, G., Xu, Y., Li, X., Li, S., Wu, P., Zhang, J., Wang, B., Bai, J., Cai, Y.-F., Cai, Z., Cao, J., Chan, ...

    work page doi:10.1007/s11433-025-2809-0 2026

  28. [28]

    and Agertz, Oscar and Berbel, Alex Agudo and Aird, James and Alexander, David A

    De Jong, R.S., Agertz, O., Berbel, A.A., Aird, J., Alexander, D.A., Amarsi, A., Anders, F., Andrae, R., Ansarinejad, B., Ansorge, W., Antilogus, P., Anwand-Heerwart, H., Arentsen, A., Arnadottir, A., Asplund, M., Auger, M., Azais, N., Baade, D., Baker, G., Baker, S., Balbinot, E., Baldry, I.K., Banerji, M., Barden, S., Barklem, P., Barth´ el´ emy-Mazot, E...

    work page doi:10.18727/0722-6691/5117 2019

  29. [29]

    In: Evans, C.J., Bryant, J.J., Motohara, K

    Tamura, N., Moritani, Y., Yabe, K., Ishizuka, Y., Kamata, Y., Allaoui, A., Arai, A., Arnouts, S., Barkhouser, R.H., Barette, R., Blanchard, P., Bergeron, E., Caplar, N., Chabaud, P.-Y., Chang, Y.-C., Chen, H.-Y., Chou, C.-Y., Chu, Y.-H., Cohen, J.G., da Costa, R.L., Crauchet, T., de Almeida, R.P., de Oliveira, A.C., de Oliveira, L.S., Dohlen, K., dos Sant...

    work page doi:10.1117/12.2628152 2022

  30. [30]

    In: Bryant, J.J., Motohara, K., Vernet, J.R.D

    Tamura, N., Yabe, K., Koshida, S., Moritani, Y., Tanaka, M., Ishigaki, M.N., Ishizuka, Y., Kamata, Y., Allaoui, A., Arai, A., Arnouts, S., Barette, R., Barkhouser, R.H., Bergeron, E., Blanchard, P., Caplar, N., Carle, M., Chabaud, P.-Y., Chang, Y.-C., Chen, H.-Y., Chou, R.C.Y., Cohen, J.G., Costa, R., Crauchet, T., Almeida, R.P., Oliveira, A.C., Oliveira,...

    work page doi:10.1117/12.3015967 2024

  31. [31]

    MUltiplexed Survey Telescope (MUST) Science White Paper I: Overview of Large-Scale Structure Cosmology in the Era of Stage-V Spectroscopic Surveys

    Zhao, C., Huang, S., He, M., Montero-Camacho, P., Liu, Y., Renard, P., Tang, Y., Verdier, A., Xu, W., Yang, X., Yu, J., Zhang, Y., Zhao, S., Zhou, X., He, S., Kneib, J.-P., Li, J., Li, Z., Wang, W.-T., Xianyu, Z.- Z., Zhang, Y., Gsponer, R., Li, X.-D., Rocher, A., Zou, S., Tan, T., Huang, Z., Wang, Z., Li, P., Rombach, M., Dong, C., Forero-Sanchez, D., Ni...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  32. [32]

    Science China Physics, Mechanics & Astronomy68(9) (2025) https://doi.org/10.1007/ s11433-025-2725-3

    Cai, Z., Huang, S., Liu, Y., Zhao, C., Huang, L.: From large telescopes to the multiplexed survey tele- scope (must). Science China Physics, Mechanics & Astronomy68(9) (2025) https://doi.org/10.1007/ s11433-025-2725-3

    work page 2025

  33. [33]

    The Jiao Tong University Spectroscopic Telescope Project

    JUST Team, Liu, C., Zu, Y., Feng, F., Li, Z., Yu, Y., Bai, H., Cui, X., Gu, B., Gu, Y., Han, J., Hou, Y., Hu, Z., Ji, H., Jing, Y., Li, W., Qi, Z., Tan, X., Tian, C., Yang, D., Yuan, X., Zhai, C., Zhang, C., Zhang, J., Zhang, H., Zhang, P., Zhang, Y., Zhao, Y., Zheng, X., Zhu, Q., Yang, X.: The Jiao Tong University Spectroscopic Telescope (JUST) Project. ...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.61977/ati2024008 2024

  34. [34]

    Journal of Open Source Software4(36), 1237 (2019) https://doi.org/10.21105/joss.01237

    Lee, G.R., Gommers, R., Waselewski, F., Wohlfahrt, K., O’Leary, A.: Pywavelets: A python package for wavelet analysis. Journal of Open Source Software4(36), 1237 (2019) https://doi.org/10.21105/joss.01237

    work page doi:10.21105/joss.01237 2019

  35. [35]

    truth table

    Earl, N., O’Steen, R., Tollerud, E., brechmos, Kerzendorf, W., Busko, I., Lim, P.L., shaileshahuja, D’Avella, D., Robitaille, T., Ginsburg, A., Homeier, D., Sip˝ ocz, B., Averbukh, J., Cherinka, B., Tocknell, J., Ogaz, S., Geda, R., Davies, J., Conroy, K., G¨ unther, H.M., Barbary, K., Cruz, K., Foster, J., Droettboom, M., Nguyen, D., Bray, E.M., Casey, A...

    work page doi:10.5281/zenodo.18473282 2000