REVIEW 2 major objections 4 minor 1 cited by
AQUAH: Automatic Quantification and Unified Agent in Hydrology
T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that one natural-language prompt can drive an entire hydrologic modeling chain—data retrieval, CREST setup, simulation, and analyst report—without any manual steps.
desk verdict A genuinely integrated LLM agent for hydrologic modeling, honest about its own limits, but the autonomy claim needs a per-basin audit of VLM-driven outlet selection and parameter initialization. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a modular agent loop wrapped around a vision-enabled large language model (VLM). The loop has four parts: an LLM interface that converts natural language into structured instructions; a tool-executor layer that calls Python geospatial libraries and model wrappers; a dynamic data pipeline that fetches DEM, precipitation, potential evapotranspiration, and discharge data; and a report-generation engine. At the center sits CREST (Coupled Routing and Excess Storage), a distributed rainfall-runoff model, configured with parameters that the VLM initializes using retrieval-augmented generation and visual reasoning. The distinctive step is on-the-fly map interpretation: instead of a human choosing the outlet and parameter values, the VLM reads the basin map and rasters and makes those choices, which is what turns a model wrapper into an autonomous agent.
What would settle it
Run the agent on a large set of basins whose true outlet and gauge locations are known from survey records, then check every run's selected outlet against the mapped stream network and every initial CREST parameter set against published plausible ranges. A nontrivial rate of silent outlet or parameter errors would refute the end-to-end reliability claim, because the pipeline gives no external signal when the visual reading is wrong.
Extended reading notes
Core claim
The central discovery is that a vision-enabled large language model can carry a hydrologic simulation from a free-form prompt to a finished report. Starting from a sentence such as "simulate floods for the Little Bighorn basin from 2020 to 2022," AQUAH autonomously retrieves digital elevation, precipitation, potential evapotranspiration, and observed discharge data; selects a basin outlet by interpreting a map; initializes CREST parameters through retrieval-augmented generation and LLM reasoning; executes the simulation; and generates a PDF with hydrographs, error metrics, and uncertainty commentary. The paper reports that across multiple basins and multiple commercial large language models, the reports were evaluated by professional hydrologists and an automated co-grader as clear, transparent, and physically plausible. The claim is explicitly scoped: the authors call for further calibration and validation before operational deployment, but they assert that the end-to-end automation itself already works.
Load-bearing premise
The entire workflow depends on the language model reading each map and elevation image correctly on the fly; if it chooses the wrong outlet or an unrealistic starting parameter set, the simulation is corrupted without any warning, and the paper does not measure how often that happens.
Editorial extensions
If this is right
- A non-expert can produce a defensible first-pass flood simulation for a basin by typing an ordinary sentence, removing data acquisition and model setup as bottlenecks.
- Rapid-response flood assessment can shift from days of manual preparation to a single prompt, since data retrieval, configuration, execution, and reporting are chained automatically.
- The generated report doubles as an interpretation layer: hydrographs, error metrics, parameter rationale, and uncertainty commentary are bundled, so the output speaks to decision-makers rather than only to modelers.
- Because the design is modular and model-agnostic, the same prompt-to-report loop can be pointed at other physics-based environmental simulators, not just CREST.
- Natural-language feedback becomes a control channel: a user can request a different gauge or different parameters and the agent reconfigures and reruns the model.
Reading between the lines
- Editorial extension: the next decisive test is a reliability audit of the visual decisions—run the agent over a large set of basins with known survey coordinates and record how often the chosen outlet or initial parameter set is wrong; the current experiments do not quantify this.
- Editorial extension: the claim that parameter initialization is physically plausible could be checked against published CONUS-scale CREST calibration distributions to see whether the agent's guesses fall inside prior ranges.
- Editorial extension: if the visual decision loop proves reliable, the same mechanism could be extended from initializing a fixed model to choosing among competing hydrologic models or planning calibration experiments, which would generalize the agent beyond cold-start simulation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces AQUAH, an LLM- and vision-based agent that takes a natural-language prompt (e.g., 'simulate floods for the Little Bighorn basin from 2020 to 2022'), autonomously retrieves DEM, precipitation, PET, and gauge data, configures the CREST hydrologic model, runs the simulation, and generates a self-contained PDF report. The architecture couples an LLM interface, a tool executor, a dynamic data pipeline, CREST model integration with RAG-informed parameter initialization, a report generator, and an interactive feedback loop. The authors report initial experiments across a range of U.S. basins, with hydrograph metrics (CC, NSCE, RMSE, BIAS) and blinded grading by professional hydrologists alongside an LLM co-evaluator. The paper frames AQUAH as the first end-to-end language-based agent for hydrologic modeling and emphasizes the role of vision-enabled LLMs in steering outlet selection and parameter initialization. The manuscript itself acknowledges that further calibration and validation are needed before operational deployment.
Significance. If the reported behavior holds up under more rigorous scrutiny, AQUAH would be a meaningful step toward lowering the barrier to physics-based hydrologic simulation: users without programming or modeling expertise could obtain analyst-ready results from a single prompt. The work is also among the first to apply vision-enabled LLMs to geospatial decision-making inside a numerical simulation loop, and its modular design could generalize to other environmental models. The paper deserves credit for including blinded human expert grading, a multi-LLM comparison, and explicit discussion of limitations. However, the central autonomy claim rests on two high-impact decisions—outlet selection and parameter initialization—that are currently not quantitatively audited, and the evaluation protocol regarding calibration versus validation periods is under-specified. The significance is therefore conditional on filling these evidence gaps.
major comments (2)
- [Section 3.1, Fig. 5] The two decisions that most determine the correctness of a cold-start simulation—outlet selection (which fixes the watershed boundary and hence the DEM/forcing extraction) and parameter initialization (which sets the starting point for calibration)—are delegated to vision-enabled LLMs, but the paper provides no per-basin audit of how often these decisions are correct. The aggregate hydrograph metrics in Fig. 5 and the ten-run variance for the Maine Coastal Basin do not rule out systematic VLM misreading, because a wrong outlet that still lies on the same river system upstream of the gauge can produce acceptable CC/NSCE after calibration. The end-to-end autonomy claim is only as strong as the reliability of these two silent steps. Please add a quantitative audit: compare the agent-selected outlet to a reference outlet for every basin (e.g., distance to gauge, upstream drainage-area error), and report the pre-calibration parameter values against physically plausible bounds for each basin and each LLM.
- [Evaluation metrics and Fig. 5 (calibration/validation protocol)] The manuscript does not clearly state whether the reported CC, NSCE, RMSE, and BIAS are computed on the same period used for calibration. Figure 5's caption refers to 'calibrated CREST parameter sets' for the best hydrographs, which suggests the evaluation period overlaps with parameter adjustment. If so, the reported skill may overstate the agent's cold-start predictive ability, since the metrics would reflect calibration fit rather than generalization. Please specify explicitly: (i) whether a separate validation period was held out, (ii) whether calibration is performed automatically inside AQUAH, and (iii) the calibration algorithm and stopping criterion. If the experiments used a split-sample design, report the corresponding validation-period metrics.
minor comments (4)
- [Abstract and Section 1] The abstract's 'first end-to-end language-based agent' claim is strong and not fully supported by the related-work survey, which only compares against RavenPy/RavenWPS and general LLM-agent frameworks. Please either soften the claim to 'one of the first' or provide a systematic search showing no prior end-to-end language-based hydrologic modeling agent exists.
- [Evaluation section (human/LLM grading)] The description of the grading methodology mentions an o3 LLM co-evaluator and blinded professional hydrologists, but it does not state the number of hydrologists, their domain experience, or how inter-rater agreement was measured. Reporting these details would strengthen the credibility of the 'analyst-ready' conclusion.
- [Throughout] The manuscript should include a data and code availability statement, including the exact versions of CREST, the LLM APIs, and any geospatial libraries, to allow reproducibility of the ten-run experiments and the grading procedure.
- [Section 2, Related Work] There is a typo 'SW AT' where 'SWAT' is intended, and the manuscript inconsistently uses 'modelling' and 'modeling' (e.g., abstract vs. Section 1). These should be harmonized.
Circularity Check
No significant circularity: AQUAH is evaluated against external observed discharge and blinded human hydrologist grading; the minor self-referential elements (CREST lineage, LLM co-evaluation) are not load-bearing reductions.
full rationale
AQUAH does not present a formal derivation chain; it is an empirical agent system whose central claim is that language-prompted, VLM-steered workflows can produce hydrologically plausible cold-start simulations. That claim is checked against external evidence: observed gauge discharge (CC, NSCE, RMSE, BIAS) and independent blinded grading by professional hydrologists. The only self-referential element in evaluation is that an LLM also scores the LLM-produced reports, but this is explicitly paired with an independent human assessment applying the same rubric, so it is not load-bearing. Self-citations to CREST and CREST-family papers are citations to an established, externally validated distributed hydrologic model and to prior calibration work; they are not used to define the current result into existence or to invoke a uniqueness theorem. The VLM outlet-selection and parameter-initialization steps are high-impact and not audited per basin, and the selection of 'best-performing' hydrographs is a reporting choice; these are evidence gaps and correctness risks, not circular reductions. No equation in the paper reduces to its own input, no fitted parameter is renamed as a prediction, and no claimed result is forced by definition. Score 2 reflects the minor self-referential grading component and the author-lineage citations, not substantive circularity.
Assumptions & free parameters
free parameters (2)
- CREST parameter initial values (LLM/RAG-inferred) =
not disclosed
- Vision/LLM decision thresholds for outlet selection and gauge matching =
not disclosed
assumptions (4)
- domain assumption The CREST hydrologic model adequately represents runoff generation and routing for the test basins.
- domain assumption The retrieved DEM, precipitation, PET, and gauge observations are accurate and consistent.
- ad hoc to paper Vision-enabled LLMs can correctly interpret maps and rasters for outlet selection and parameter initialization.
- domain assumption Hydrologist expert ratings are a valid measure of report quality and physical plausibility.
Cite this review
Pith. "Pith review of AQUAH: Automatic Quantification and Unified Agent in Hydrology." pith.science (2026). https://pith.science/paper/2VPXXQJ3
@misc{pith2026250802936,
author = {Pith},
title = {Pith review of: AQUAH: Automatic Quantification and Unified Agent in Hydrology},
year = {2026},
howpublished = {\url{https://pith.science/paper/2VPXXQJ3}},
note = {Machine review of arXiv:2508.02936}
}
read the original abstract
We introduce AQUAH, the first end-to-end language-based agent designed specifically for hydrologic modeling. Starting from a simple natural-language prompt (e.g., 'simulate floods for the Little Bighorn basin from 2020 to 2022'), AQUAH autonomously retrieves the required terrain, forcing, and gauge data; configures a hydrologic model; runs the simulation; and generates a self-contained PDF report. The workflow is driven by vision-enabled large language models, which interpret maps and rasters on the fly and steer key decisions such as outlet selection, parameter initialization, and uncertainty commentary. Initial experiments across a range of U.S. basins show that AQUAH can complete cold-start simulations and produce analyst-ready documentation without manual intervention. The results are judged by hydrologists as clear, transparent, and physically plausible. While further calibration and validation are still needed for operational deployment, these early outcomes highlight the promise of LLM-centered, vision-grounded agents to streamline complex environmental modeling and lower the barrier between Earth observation data, physics-based tools, and decision makers.
Forward citations
Cited by 1 Pith paper
-
An Agentic Interface for End-to-End Probabilistic Seismic Hazard and Risk Analysis
An MCP-based LLM-orchestrated server provides end-to-end European seismic hazard and risk calculations, reproducing official ESHM20 475-year spectra to a median of about 5%.
Reference graph
Works this paper leans on
-
[1]
Anthropic. Claude sonnet 4: A hybrid-reasoning ai model. Technical Report / Model Card claude-sonnet-4@20250514, Anthropic, Inc., San Francisco, CA, 2025. Mid-sized model in the Claude 4 family; 200 K token context, extended “thinking” mode, strong coding and reasoning capabilities, safety at AI Safety Level 2. 2, 5
work page 2025
-
[2]
Richard Arsenault, David Huard, Jean-Luc Martel, Mag- ali Troin, Juliane Mai, François Brissette, Christian Jauvin, Long Vu, James R Craig, Trevor J Smith, et al. The pavics- hydro platform: A virtual laboratory for hydroclimatic mod- elling and forecasting over north america. Environmental Modelling & Software, 168:105808, 2023. 2
work page 2023
-
[3]
Abdolreza Bahremand. Hess opinions: Advocating process modeling and de-emphasizing parameter estimation.Hydrol- ogy and Earth System Sciences, 20(4):1433–1445, 2016. 1
work page 2016
-
[4]
Terramesh: A plane- tary mosaic of multimodal earth observation data
Benedikt Blumenstiel, Paolo Fraccaro, Valerio Marsocci, Johannes Jakubik, Stefano Maurogiovanni, Mikolaj Cz- erkawski, Rocco Sedona, Gabriele Cavallaro, Thomas Brun- schwiler, Juan Bernabe Moreno, et al. Terramesh: A plane- tary mosaic of multimodal earth observation data. In Pro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 2...
work page 2025
-
[5]
Using expert and non-expert models of climate change to enhance communi- cation
J Stuart Carlton and Susan K Jacobson. Using expert and non-expert models of climate change to enhance communi- cation. Environmental Communication, 10(1):1–24, 2016. 1
work page 2016
-
[6]
Conus- wide model calibration and validation for crestv3
Mengye Chen, Zhi Li, Humberto J Vergara, Jonathan J Gour- ley, Ming Xue, Yang Hong, Xiao-Ming Hu, Hector Mayol Novoa, Elinor R Martin, Renee A McPherson, et al. Conus- wide model calibration and validation for crestv3. 0–an im- proved coupled routing and excess storage distributed hydro- logical model. Journal of Hydrology, 626:130333, 2023. 11
work page 2023
-
[7]
The ensemble framework for flash flood forecasting (ef5) v1
Zachary L Flamig, Humberto Vergara, and Jonathan J Gour- ley. The ensemble framework for flash flood forecasting (ef5) v1. 2: Description and case study. Geoscientific Model Development, 13(10):4943–4958, 2020. 3, 11
work page 2020
-
[8]
Shang Gao, Zhi Li, Mengye Chen, Peirong Lin, Zhen Hong, Daniel Allen, Thomas Neeson, and Yang Hong. Spatiotem- poral variability of global river extent and the natural driving factors revealed by decades of landsat observations, grace gravimetry observations, and land surface model simula- tions. Remote Sensing of Environment , 267:112725, 2021. 1
work page 2021
Show all 27 references
-
[9]
Applications of the swat model special section: overview and insights
Philip W Gassman, Ali M Sadeghi, and Raghavan Srini- vasan. Applications of the swat model special section: overview and insights. Journal of Environmental Quality , 43(1):1–8, 2014. 2
2014
-
[10]
Gemini 2.5 flash: A hybrid reasoning ai model
Google DeepMind. Gemini 2.5 flash: A hybrid reasoning ai model. Technical Report (Model Card) gemini-2.5-flash- preview-05-20, Google LLC, Mountain View, CA, 2025. Preview version available via Vertex AI / AI Studio; sup- ports controllable reasoning with 1M token context and ...
2025
-
[11]
Terramind: Large-scale generative multimodal- ity for earth observation
Johannes Jakubik, Felix Yang, Benedikt Blumenstiel, Erik Scheurer, Rocco Sedona, Stefano Maurogiovanni, Jente Bosmans, Nikolaos Dionelis, Valerio Marsocci, Niklas Kopp, et al. Terramind: Large-scale generative multimodal- ity for earth observation. arXiv preprint arXiv:2504.11171,
-
[12]
A survey on llm-based multi-agent systems: workflow, infras- tructure, and challenges
Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. A survey on llm-based multi-agent systems: workflow, infras- tructure, and challenges. Vicinagearth, 1(1):9, 2024. 2
2024
-
[13]
Crest-imap v1
Zhi Li, Mengye Chen, Shang Gao, Xiangyu Luo, Jonathan J Gourley, Pierre Kirstetter, Tiantian Yang, Randall Kolar, Amy McGovern, Yixin Wen, et al. Crest-imap v1. 0: A fully coupled hydrologic-hydraulic modeling framework dedicated to flood inundation mapping and prediction. En-...
2021
-
[14]
The con- terminous united states are projected to become more prone to flash floods in a high-end emissions scenario
Zhi Li, Shang Gao, Mengye Chen, Jonathan J Gourley, Changhai Liu, Andreas F Prein, and Yang Hong. The con- terminous united states are projected to become more prone to flash floods in a high-end emissions scenario. Communi- cations Earth & Environment, 3(1):86, 2022. 1
2022
-
[15]
A decadal review of the crest model family: Developments, applications, and out- look
Zhi Li, Xianwu Xue, Robert Clark, Humberto Vergara, Jonathan Gourley, Guoqiang Tang, Xinyi Shen, Guangyuan Kan, Ke Zhang, Jiahu Wang, et al. A decadal review of the crest model family: Developments, applications, and out- look. Journal of Hydrology X, 20:100159, 2023. 1, 11
2023
-
[16]
Deepseek-v3 technical report
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. 2
2024 arXiv
-
[17]
Dilemmas in developing models for long-term drought risk management: The case of the national water model of the netherlands
Marjolein Mens, Bennie Minnema, Koen Overmars, and Bart Van den Hurk. Dilemmas in developing models for long-term drought risk management: The case of the national water model of the netherlands. Environmental Modelling & Software, 143:105100, 2021. 1
2021
-
[18]
Gpt-4o system card, 2024
OpenAI. Gpt-4o system card, 2024. 2, 5
2024
-
[19]
Kosmos-2: Ground- ing multimodal large language models to the world
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Ground- ing multimodal large language models to the world. arXiv preprint arXiv:2306.14824 , 2023. Also described in Mi- crosoft Research technical report. 2
2023 arXiv
-
[20]
Gpt-4 is here: what scientists think
Katharine Sanderson. Gpt-4 is here: what scientists think. Nature, 615(7954):773, 2023. 2
2023
-
[21]
Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:38154–38180,
-
[22]
Prithvi-eo-2.0: A versatile multi-temporal foundation model for earth observation ap- plications
Daniela Szwarcman, Sujit Roy, Paolo Fraccaro, Þorsteinn Elí Gíslason, Benedikt Blumenstiel, Rinki Ghosal, Pedro Hen- rique de Oliveira, Joao Lucas de Sousa Almeida, Rocco Sedona, Yanghui Kang, et al. Prithvi-eo-2.0: A versatile multi-temporal foundation model for earth observa...
2024
-
[23]
Gemma: Open models based on gemini research and tech- nology
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and tech- nology. arXiv preprint arXiv:2403.08295, 2024. 2
2024 arXiv
-
[24]
The coupled routing and excess storage (crest) distributed hydrological model
Jiahu Wang, Yang Hong, Li Li, Jonathan J Gourley, Sadiq I Khan, Koray K Yilmaz, Robert F Adler, Frederick S Poli- celli, Shahid Habib, Daniel Irwn, et al. The coupled routing and excess storage (crest) distributed hydrological model. Hydrological sciences journal, 56(1):84–98,...
2011
-
[25]
Pcssr-dnnwa: A physical constraints based surface snowfall rate retrieval algorithm us- ing deep neural networks with attention module
Songkun Yan, Ziqiang Ma, Xiaoqing Li, Hao Hu, Jintao Xu, Qingwen Ji, and Fuzhong Weng. Pcssr-dnnwa: A physical constraints based surface snowfall rate retrieval algorithm us- ing deep neural networks with attention module. Geophysi- cal Research Letters, 50(13):e2023GL103923, 2023. 2
2023
-
[26]
React: Synergizing rea- soning and acting in language models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing rea- soning and acting in language models. In International Con- ference on Learning Representations (ICLR), 2023. 2
2023
-
[27]
I want to simulate the streamflow of the Mad–Redwood basin from 2020 to 2022
Siyu Zhu, Zhi Li, Mengye Chen, Yixin Wen, Shang Gao, Jiaqi Zhang, Jiao Wang, Yi Nan, Sebastian C Ferraro, Theresa E Tsoodle, et al. How has the latest imerg v07 improved the precipitation estimates and hydrologic utility over conus against imerg v06? Journal of Hydrology, 645:...
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.